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EXECUTIVE SUMMARY 



This report summarizes the national implementation and impact of the Emcirgenry 
School Aid Act (ESAA) Basic and Pilot Programs during the 1973-74 school year 
— the first year of their operation in school districts across the nation- 
The ESAA Basic and Pilot programs are both elementary and secondary School pro- 
grams designed to deal with problems associated with minority-group isolation. 
The Basic program is a desegregation program that provides funds to local school 
districts (a) to encourage their elimination of minordty-group ii;olation, (b) to 
help them meet special needs^ incident to the elimination of segregation and dis- 
crimination; and (c) to help them overcome educational disadvantages associatea 
with minority-group isolation. In contrast to the Basic program, the ESAA Pilot 
program is essentially a compensatory education program for students enrolled 
in minority-isolated schqbls (i.e., schools Vith 50% or greater minority enroll- 
ment) , ' ^ • 

The national evaluatior>' of the Basi^ and Pilot programs was designed to determine 
-^'^^i^cumulative impact of ESAA programs in terms of the Act's objectives over a 
peri5^d of three school years. The evaluation design anticipated the fact that 
the first year of imjolementation of any national educational program is always 
a formative one, requiring adaptation of schools, school staffs, and students 
to new projects. Consequently, this report, which .focuses on year one of ESAA, 
should be considered a progress report on that process of implementation and 
adaptation rather than a definitive report on the success of the Act in achieving 
its objectives. Cater ESAA evaluation reports should provide the measure for 
assessing the Act>*s effectiveness. 

Although preliminary in nature and premature for assessment of ultimate program 
effectiveness, /the first7year ESAA evaluation results are encouraging. First, 
the findings j^hdicate that the assumption upon which the legislation was based 
is a matter of fact — minority-group isolation in our nation's schools is indeed 
associated with educational disadvantages. Students in minority-isolated schools, 
regardless ^f their racial or ethnic status, are achieving significantly below 
the national norm in reading and mathematics. Further, studepts in minority- 
isolated schools, or recently from such schools, tend to be more educationally, 
socially^*" and economically disadvantaged than their peers in non-isolated schools. 

A secqrtd finding, related to the first, is that students in minority-isolated 
schoc^is and many students in desegregated schools have needs associated with 
thei^ educational and economic disadvantagement that are directly related to the 
ob<jectives of ESAA. Even more relevant for a preliminary assessment of program 
implementation, first-year evaluation results seem to indicate that ESAA resources 
/in terms of both dollars and services have, -on the \^hole,-been properly targeted 
at those documented needs. There is also some evidence that the resource alloca- 
tion process began to have positive impact on the academic achievement of students 
in ESAA-funded districts over the short five-and-one-half-month period evaluated. 
In short, although preliminary, the findings of the first year of evaluation 
suggest that the ESAA program is based upon sound assumptions and may ultimately 
achieve one or more of its objectives. 
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In addition to the abovfe national impact findings, the first year-results 
suggest that relationships among student, staff, and program? characteristics 
and student outcomes differ somewhat for minority-isolated as compared to 
desegregated schools. In, minority-isolated schools, preliminary findings 
indicate that there is a positive relationship between the level of supple- 
mental reading program funding and student reading and mathematics achievement. 
Similarly, lower pupil/teacher ratios seem to be positively related to student 
mathematics achievement in minority-isolated schools. For reasons that are 
not clear, no such relationships were found in desegregated schools. Never- 
theless, in desegregated schools, positive desegregation-related policies ^ 
attitudes, and activities of district and school staff appearrelated to 
positive student expectations, student liking for school, .an^^udent reading 
and mathematics achievement. Also in desegregated schools, at least at the 
secondary level, the results suggest a positive relationship between reading 
achievement and the amount of time spent in reading instruction. Findings such 
as the above, if validated by second- and third-year results, should provide 
guidance for lejjislative and/or regulatory changes designed to increase program 
effectiveness. 

As the reader will find upon reviewing this report, the above findings are 
not definitive and should not be construed as final conclusions regarding 
the Act*s effectiveness — the data for such an assessment will be summarized 
in ^he evaluation teports for years two and three. Nevertheless, the prelim- 
inary findings are encouraging, and will form an analytic foundation upon which 
the primary impacc: assessment of later reports can be properly grounded. 



MichaeljtJ. Wargo, Ph.D. 

U.S. Ofrice of Education 

ESAA Evaluation Program^ Officer 
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CHAPTER I 



INTRODUCTION 



Under contracts from the U.S. Office of Education, System Development Cor- 
poration (SDC) is conducting an evaluation of two closely related programs 
authorised by the Emergency School Aid Act (ESAA)\ The Pilot program was 
created to support "unusually promising ESAA pilot programs or projects designed 
to overcome the adverse effects of minority group isolation by improving the 
academic achievement of children," Eligible districts must have 15,000 
minority students, enrolled, or minority students must constitute more than 
50% of the total enrollment. The Basic Grants program consists of grants to 
local education agencies for the purpose of implementing plans to (a) com- 
pletely eliminate minority isolation in all schools within the agency, (b) 
eliminate or reduce minority isolation in one ot more schools within the agency, 
(c) reduce the total number of minority-group children who are in isolated 
schools, or (d) prevent minority isolation that is reasoncibly likely to 
occur witnout assistance under the Act, Basic Grant applicants have been 
encouraged to focus their programs on improving basic educational skills. 

The combined Basic/Pilot evaluation involves collection of data over a period 
of three school years, 197 3^-74, 1974-75, and 1975-76, This report describes 
the evaluation results for the first year (1973-74), Subsequent reports will 
present results for 1974-74 and 1975-76, as well as cumulative trends over 
the two- and three-year periods. 

For reasons described in the report, the only criterion measures of program 
success available for the 1973-74 evaluation were students' scores on 
standardized achievement tests in reading and matliematics. In the next two 
evaluation years, increased emphasis will be placed on desegregation-related 
activities and outcome measures. Specifically, test scores will be supple- 
mented by criterion measures of reduction in minority-group isolation and by 
indicators of desegregation-related school climate. 

It is anticipated that the report may have several different audiences, each 
with its own particular interests and knowledge. These potential audiences 
include Congress and the Office of Education, eivxl rights and other 
advocate groups, district-level program administrators, educational researchers, 
and evaluation specialists. Because the primary goal of the evaluation is to 
inform decision-makers, rather than to add to the literature on evaluation/ 
research methodology, the major focus tliroughout the report is on results, not 
techniques. Excessive procedural details have been avoided, and, where 
possible, major findings have been highlighted. Most of the data tables have 
been relegated to the appendix and are referred to in the appropriate pages 
of the text. 
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The remainder of this chapter provides a brief overview of the operational 
ESAA programs and of the evaluation objectives and methodology, and discusses 
certain limitations of the present report. Chapter IX. describes the rationale 
ijind methodology of the evaluation sampling procedures and presents data con- 
cerning the representativeness of the resulting samples. \Chapters III through 
VI describe the school programs and funding alj^ocations, the community/district 
context within which the sample programs operated, and -the sample students 
enrolled in those programs. Chapter .VIl focuses specifically on the de3Bgre- 
gation plans and activities of tlie ^ainple Basic districts and examines evidence 
concerning the amount of district desegregation possibly associated with ESAA 
program implementation. Chapters /^lyaijd JX discuss achievement-test gains 
made by sample students during th^ 1973-74' school year and attempt to.-relate^ — 
those, gains to the^ experimental variable (ESAA- funding) and to other student 
and program dimensions. Finally^ Chapter X presents.'an overall sximmary of the 
major findings and conclusions for the first^ evaluation year. 



A. OVERVIEW OF ESAA PROGRAM LEGISLATION AND FUNDING 

The Emergency School Aid Act (ESAA) 'was enacted into law in June of 1972 to 
provide elementary and secondary school districts with financi'^'l assistance 
to (1) meet the special needs incident to the elimination of minority-group 
segregation and discrimination; (2) encourage the voluntary reduction, 
elimination, or. prevention of minority-group isolation; and (3) aid children 
in overcoming the educational disadvantages of minority-group isolation 
(P.L. 92-318, Sec. 702 (b) J ; Mhlle the Act as amended , in/l974 (P.L. 93-380, 
Sec. 641) authorizes the appropriation of $1 billion for fiscal year 1973 and 
a similar amount for the period ending June 30, 1975, actual appropriations 
h^ve amounted to $270 millioM, $234 million, and $215 million for fi.scal years 
1973, 1974, and 1975, respeccively. Since funds are annually appropriated for 
obligation and expenditure during the fiscal year succeeding ^he year of 
appropriation, the major thrust of the Act began durin-^ school year 1973-74 
and is expected to continue through school year 1976-7/7. 

Seventy-four percent of the Act's annual apprjppriation is reserved for two 
subprograms, the Basic Grants program (59%) and the Pilot prograifi (15%). The 
Basic Grants pilogram, is essentially a desegregation program designed to reduce 
minority-group I isolation, to meet the needs incident to the elimin^ion of 
segregation and discrimination, and to aid school children in^ bver9oming the 
educational difcadvantages of minority-group isolation.. In ppntrast, the 4>iiot 
program is a compensatory education program designed to improve the •academic 
achi^ivement of children in minority-isolated schools (i.e., schools with oyer 
50% minority enrollment) . ' ^ 

The sums annually appropriated pursuant to the Act arc apportioned to states 
on the basis of the ratio of their number of minority-group* school-aged 
children to the number of such children in all states. Local school districts 
compete for the funds apportioned to their state through *grant applications to 
their HEW Regional Office. In aj^plying for an ESAA grant, a local school ' j 
district must demonstrate that it has needs ^elated to the Act's objectives' 
and that it has designed a program based on authorized activities that show 
promise of achieving one or more of the Act's objectives. 
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B. DESCRIPTION OF ESAA EVALUATION 

The Act authorizes a national evaluation of its programs which may be supported 
by up to an annual* 1% reservation of appropriated ESAA funds. As designed by 
the U.S. Office of Education (USOE) and conducted by System Development Corpo- 
ration (SDC)y the national evaluation focuses on the integrated evaluation of 
the ESAA Basic and Pilot programs. The remainder of this section summarizes 
the major evaluation objectives for both programs and describes the general ' 
methpdology being applied to meet those objectives. 

1. EVALUATION OBJECTIVES 

The overall evaluation has several major objectives, some of which involve 
longitudinal analyses and comparisons of data collect-ed over two or three 
years. Certain other objectives involve data collected in , special ^studies not 
conducted until the second evalua1:ion year. A summarized list of all the key 
study objectives is presented belowr each objective is accompanied by a brief 
note concerning the extent to which that objective is addressed in the present 
first-year report." > 

• Determination of the short-- and long-term national impact of the program 
in terms of the ,Congressionally authorized program objectives, namely, 
reduction* in minority-^group isolation, elimination of discrimination, and 
^improvement of basic^ skills in ej^jmentary and secondary schools. The 
present report is based on a single year's impact data, and therefore is 
strictly cross-sectional in nature. Furthermore, the oAly type of outcome 
(criterion) measure used for gauging program impact in the first year was 
students" performance on standardized achievement tests. Longitudinal 
trends.' and 'outcome d^ta based on measures of minority-group isolation and 
pISVceived school discrimination will be discussed in the Year Two and Year 
Three Reports. 

• Identification and description of the needs of students in or from minority- 
isolated schools; the characteristics of local programs, including the 
relationships between student needs and resource allocations in the sarAple 

, districts; and the interrelationships of student needs, program characteristics, 
and program impact. This objective is directly addressed in the present report. 

• Documentation and dissemination of information relating to unusually 

successful^ local programs and program components that appear to be related 

to success. In-depth data on successful sites were not collected until the 

second evaluation year and are not included in this report. 

V 

• Determination of the relative effectiveness of three foinns of educational 
injtervention — desegregation, canpensatory education, and a combinat^n of 
these — as compared to minority-isolated schools with no special "intervention. 
These comparisons have been deferred until the second-year report to allow a 
longer period for the different program approaches to show differential iTnpacts. 

• Investigation , of the relationships among regular school expenditures,* 
supplemental ESAA expenditures, and program impact, in an attempt to determine 

^ local program cost-effectiveness and" the minimum supplemental expenditures 
necessary to ensure some measure of program success*. The present report 
examines relationships between per-pupil funding levels (regular and 
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supplemental) and program impact, but the determination of minimum necessary 
supplemental expenditures ("critical mass") is deferred to the second-year 
report to allow the inclusion of more detailed in f cybernation on resource 
allocations in successful sites* 



2, GENERAL METHODOJiCGY 

The combined Basic/Pilot evaluation involves collection of data over a period of 
three years, beginning with the 1973-74 school year addressed in this report. 
This multi-year design allows analyses of cumulative program impact and of 
possible fadeout of program impact 'after students are transferred out of the 
ESAA activities. In addition, it provides an opportunity' to assess program 
maturation effects (e.g., greater program impact on each successive wave of 
new students in the second and third years of operation) . 

For both Pilot and Basic programs, standardized achievement tests are- administered 
at the beginning and end of each school year to assess gains made in reading 
and math by a sample of students participating in th*e ESAA projects. The same 
tests are given to a sample of students in control (non-ESAA) schools, which are 
drawn' at random from the same districts as thfe treatment (ESAA-'funded) sample 
schools and selected to match as closely as possible the characteristics of 
their paired treatment schools. This eihphasis on matched schools within districts 
makes it possible to hold important district factors common in paired comparisons 
of treatment and control schools and to achieve a true experimental design by 
random assignment of each pair of schools to the treatment and control conditions. 

Because of interest in evaluating ESAA programs at both the elementary and 
secondary levels, schools at both levels are included in the Basic Grant 
program sample. Too few Pilot awards weiTe given at the secondary level to 
include high schools in the Pilot sample. Grade levels included in the evaluation 
are grades 3, 4, and 5 (elementary) and 10, 11, and 12 (secondary). The ^use 
of three successive grades at each school level allows longitudinal following 
of individual sample students for up to three years, depending on their grade 
placement at the start of the evaluation. To be eligible for inclusion in 
the study, schools must have sufficiently large numbers of ESAA participants 
(or ESAA-eligible studeftts, in the case of control schools) to allow for 
attrition over the evaluation period; toward this end, larger samples were 
initially drawn at grades 3 and 10 than at other grades to allow for following 
those samples for three years. 

Two other major outcome measures planned for the multi-year evaluation consist 
of data on perceived discrimination (school climate) and on reduction of 
minority-group isolation. It was not possible to obtain adequate data on these 
variables for use as criterion measures in the present report,' but they will be , 
included in reports for the second and third years. 

In. addition to the outcome measures, questionnaires and other recording forms 
are used once each year to obtain extensive data on the school programs them- 
selves (in both treatment and control .schools) and on sample students. Near 
the end of each school year, a battery of questionnaires is administered to 
superintendents, district business managers, local ESAA coordinators, principals, 
teachers, and students in the sample. Those questionnaires provide data on 
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district, school, and classroom minority-group isolation, program operation, 
resource allocation, and student and staff background characteristics. 

Finally, student activity logs are used to record the amount of time that a 
student is exposed to different types of educational experiences (e.g.^ peer 
tutoring in math, cultural enrichment programs, etc.). Th^ logs provide a 
cumulative record of each student's interactions with the educational system, 
with emphasis on compensatory activities of the sort presumably stressed by 
ESAA projects. 

Data analyses, designed to focus on the major evaluation goals described above, 
include determinations of pretest-posttest changes in outcome measures, to 
assess overall gains across both treatment and control schools? comparisons of 
pretest-posttest gains between treatment and control schools in the matched 
pairs, to evaluate the impact of the experimental variable (ESAA funding) ; 
comparisons of outcomes associated with different intervention approaches; 
ranking of lo^l programs with respect to impact, to identify particularly 
successful pr^ects; and analyses of relationships among program features, 
student characteristics, and program impact. In connection with the second-year 
evaluation, school expenditure data will be analyzed in relation to impact data 
to determine the minimum supplemental expenditures necessary to ensure program 
success and to identify local project componei\ts that differentiate successful 
from unsuccessful projects. 



C. REPORT LIMITATIONS * 

Readers of this report should be aware of several important limitations stepmung 
from the short period of ESAA program operation covered by the report and from 
constraints in the types of data available during the initial evaluation year. 

Seme of the local programs included ^n the first-year evaluation were not 
in full operation until at- least two \>x three months into the school year, 
with the result that districts often ^tould not identify ESAA-participatmg 
students until late October or Novejpl^rer 1973. This, in turn, led to a pretest 
schedule that stretched into November or, in a few cases, into December. As a 
consequence, the average^'"pretest-posttest interval was only slightly more than 
five months. Such a brief period of time can hardly be considered a true test 
of the effectiveness of the ESAA program, or of any other innovative educational 
program seeking to effect broad improvements in schools with minority-isolated 
and disadvantaged students. 

A second liijiitation in the first-year evaluation and in this report is the fact 
that only one type of out;come measure — achievement test performance — could 
be used in the analyses o^ program impact. This constraint is clearly an 
important one, considering that !key objectiyes of the ESAA Basic program also 
include reduction of minority-group isolation and in-school discrimination. 
An attempt was made in the first year to collect pretest and posttest data on 
racial attitudes by means of a questionnaire designed for administration to 
studen1:s at the beginning and end of the school year. The questionnaire was 
given at the start of the year, but strong opposition to the instrument was c 
raised by several districts, resulting in a federal government decision not to 
release the questionnaire for end-of-year administration. Thus, it vas 
impossible to assess race-related attitudinal changes that might have occurred t 



during the school year, subsequently, a new instrument has been developed by 
a special advisory panel to t±e ESAA evaluation project, consisting of members 
of various minority-group and other interested organizations. This instrument, 
which focuses on students* perceptions of systematic school discrimination, 
was administered at the beginning and end of the second evaluation ye^r. It 
.w[ill provide the basis for including a second type of outcome measure (school 
ciiiaate) for assessing program impact in later evaluation reports- 

With respect to the third intended category of outcomes, reduction in minority- 
group isolation, Office of Civil Rights (OCR) data pertiment to this outcome 
dimension were analyzed, and descriptive summaries are provided in. Chaptet VII 
of this report. However, because of certain internal inconsistencies in, some 
of the data, and because the indices used by OCR as measures of desegregation 
are somewhat insensitive to the types of changes that might be expected in this 
study, it was decided not to use these data in analyses of Basix: progr/am impact 
for the first evaluation year- Efforts are currently under way to develop 
improved measures for yse in subsequent reports, ^' 

One further limitation of the report is that it represents only a cross-sectional 
view of a single year of program operation. Some of the major eykluation 
objectives deal with multi-year/ longitudinal effects of the ESA^ programs. 
Analyses pertaining to these important effects will be included' in later reports 
on the second and third evaluation years. 



CHAPTKR II 



EVALUATION SAMPLE , SELECTION 

Implicit in the ESAA evaluation objectives^ (see Chapter I) were several important^ 
requirements for the sample desigiK The mpst critical of these requirements 
were as follows: ^ 

• Evaluation data were needed at both .the elementary an^ secondary 
levels, as there was strong interest in ESAA program (operations 
and impact at both leve},s,^ - . t^^-- 

• The longitudinal nature of th6 s,tudy required data collection at 
multiple grade levels, so thpit cohorts of students starting ESAA 
participation at different times and grades could be followed for 
more than one year. To meet this requirement, three -consecutive 
grade levels were selected at the elementary level (grades 3 through 
5) and three at the secondary level (grades 10 through 12). 

• Sufficiently large sample? of students were needed at each grade 
level in each sample scKool to allow for anticp-ipated attrition over 

"^^^ pefio"d'~af~Tif> to three-yea.rs-_ ^ ^ 5i — 

• The sample universe was limited where possible to districts having 
two or more schools designated for EfSAA participation. By this 
means, matched pairs of treatment and control schools could be 
sought with total intra-district. equating of district-level variables- 

A, DEVELOPMEM^M?!^ SAMP L'fe UNIVERSE DEFINITION 

_ — ^ 

The initiial Sample universe was derived from districts receiving ESAA awards 
for school year 1973*1974 in April or June 1973. This liniverse was further 
limited to districts having Bas^ic and/or Pilot programs planned for gi^ades 3, 
4/ and 5, or for grades 10, 11, and 12, , - 

At this point, a* preliminary review was conducted of district grant applications 
to determine how many districts were likely to meet the sample ..universe require-' 
ments^^as initially defined^ Data available in most applications included the 
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names of ESAA-^articipating* schools, participating grade levels, numbers of 
•participating students, percentagss and types of participating minority students 
and types of ESAA activities planned. 

Based/in part on datA in the district grant applications, and in part on design 
requirements discussed above, a preiviminary Sampling pl^n was adopted^ subject 
to final' revision as Iftirther data bec^ame available on the ESAA programs. Major 
features bf the plan were as follows :\ . * ' 

\ 

• Three independent universes woul3 be defined and sain|)led: a Pilot 
^ elementary universe, a Basic ^lementary universe, an^'a B^sic 

secondary universe. No Pilot l^econdary schools -would be considered, 
since the number of Pilot awarcls involving secondary schools was small. 

• The Pilot and Basic elementary samples would include students at 
grades 3^, 4, and 5, while the Basic secondary sample would include 
grades 10, 11, and 12. 

• No sample would be sought from "overlap" districts receiving both 
Basic and Pilot grant awards. Althaugh it was initially planned to 
give special attention to this "overlap" universe, the number of such 
districts was found- insufficient for analytic purposes; given this 
constraint ,r . it was felt that the inclusion -pf /some districts with one 
type of program^ and some with both types wbuJ,d unnecessarily confound 
the evaluation design. / ^ 

- > _ \, 

• For each of the 'thVee universes^ a sample of at least 25 pairs^ of 
participating schools would be sought. Schools would be paired by 
similarity rin -minority-group enrollment, in pre-ESAA achievement level, 
and in sobipeconoraic status. VJithin pairs, treatment and controi\ 
conditions would be randomly assigned; this would result in ESAA ftands 
being withheld from, one school of each selected, "participating" pai^ of 
schools . . 

• Students would be sampled across classes, rather than by intact classes, 
to increase the nymber of classes and teachers represented within the 
sampling constraints. . 



For purposes of this sampling discussion, reFerences to schools' or students* 
"participating" in ESAA activities simply means, that they were so designated 
in the districts* grant applications. In reality, none of the schools or 
students were actively engaged in ESAA activj^ties at the time of sample 
selection; furtliermore , some of them were ultimately assigned to the control 
(non-ESAA-fundedl condition, and therefore never participated in ESAA programs. 
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To implement the sampling plan and to further define the desired universes, 
additional information Was obtained by telephone from the school districts. Prior 
to telephone contact, a screening procedure was initiated to reduce the number of 
calls. Pilot applications were screened to eliminate all districts not' having 
planned ESAA activities in'grade 3, 4, or 5, and Basic applications were reviewed 
to exclude all districts not having planned ESAA, activities in any of the grade 
levels of interest, A further reduction of the/Basic elementary group was 
accomplished by removing a 50% random sample from the set of districts Having 
comparatively small numbers*of participating schools , or having ESAA activities 
in only one or two jof the grade levels of i/iterest. This reduced the total 
number of required calls ±o 259 districts. In these calls, each district was 
asked for information on the name of each schqol designated to participate in 
ESAA, the participating grade levels within each school, the type and percentage 
of each minority group in the school's enrollment, comparative ratings within the 
district of each school's academic achievement, and comparative ratings within the 
district of the school's socioeconomic status^ j 

The rating of socioeconomic status was a comparative estimate by the distripfc 
6f the status of the families served by each school. This estimate was based 
on such factors as the school's participation in Federal school lunch programs. 
The index of academic achievement was based in most cases on school records of 
standardized achievement test scores, / 

'\ r ' ^ . 

B. ^ EVALUATION UNIVERSES AND SAMPLES, REDEFINED TQ REFLECT PROGRAM DATA 

After redefinition to reflect data obtained gn the planned programs in ESAA 
districts, the evaluation universes and samples were as follows; 

1. PILOT ELEMENTARY SAMPLE 

BecausG of constraints in the number of funded Pilot programs, it was necessary 
to select a primary Pilot evaluation universe and a supplemental evaluation 

universe. The priqiary evaluation universe consisted of all districts with^ 

Pilot programs awarded in April or June 1973, having two or more participating 
schools with Pilot activities in at least two consecutive grade levels of 
interest. Additional requirements of this universe were (1) that the 
participating schools should be pairable (i.e., similar in student socip- 
economic status, pre-ESAA achievement, and percentage and type of minority 
enrollment) , and (2) that each participating school should have at least 
25 participating students in each of grades 3; 4, and 5: 

Only 17 Pilot districts could be identified that met the requirements o^ the 
primary evaluation univer/se. All of these districts were included in the final 
sample, as it was originally desired to have at least 25 districts with pairable 
schools. Therefore, at the district level, the sample was identical to the 
"primary evaluation universe. Each of these districts became a stratum, with the 
primary sampling unit being the school. One school was randomly selected With 
equal probability within each district. The school that was the "best possible" 

II-3, 



20^ 



match was then selected by employing the above crit^'ria. One school in this pair 
was randomly assigned to the treatment condition andl the other automatically 
became the control school for that district, ; 

\ » 

Because of the small number of resulting pairs, it was felt necessary to select 
additional Pilot schools that could not be paired, (IJhese ESAA--participating 
schools were to be used to round out the sample for descriptive purposes, but 
could not be used, of course, in the experimental analysis cf treatment-control 
differences in acMavement test scores.) Thus a second, or supplemental 
evaluation universe was defined to augment the Pilot sample. This supplemental 
universe initially consisted of Pilot districts having unpairable schools 
with Pilot programs in at least two consecutive grades, of interest. Each district 
containing one or more acceptable schools was designated, as a stratum. One 
school was randomly selected with equal probability within each stratum and 
designated as a treatment school. This resulted in 18 additional treatment 
sdhools being added to the sample. 

To further increase the Pilot sample, there was a second round of sampTirig from 
districts in the supplemental universe, A district was resampled if i€ had 
either or both of the follow'ing conditions: (1) one or more previously un- 
pairable schools; (2) one or more previously unselected pairs of schools. Each 
district from this group was designed as a stratum, and one school was randomly 
selected from each stratum. This process resulted in 10 additipnal treatment 
schools- 

In districts in which the grades of interest were not housed in the same school 
(for example, where grade 3 was in a primary school and grades 4 and 5 were in 
a middle school) , the two schools were treated as a single school unit in the 
sample, 

A total of 35 districts were represented in the final Pilot sample. This 
included 17 districts with matchtid pairs of treatment and control schools 
(i,e,, districts in the primary evaluation universe), and 28 distificts with a 
single treatment school. (Ten of the districts had both a treatment-control pair 
and a single treatment school,) Of the total 62 schools, . therefore, 45 were in 
the treatment condition, * 

2, .BASIC ELEMENTARY SAMPLE / ' ^ 

The Basic elementary evaluation universe consisted of districts receiving Basic 
elementary grants in April or June 1973, having two or more pairable schools 
with Pilot program activities designated for grades 3, 4, and 5, An additional 
requirement was that each participating school should have designated at least 
50 participating students in each of those grades. 

Because it was evident that there would be more than an adequate number of 
districts meeting these critei^ia, a further reduction was ihade in the uniyerse 
of eligible districts by eliminating a 50% random sample from .the set of districts 
having comparatively small numbers of participating schools or having only one or 
two of the grade levels of interest. The resulting universe was found to inclu<?e 
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47 Basic elementary districts, all of which were selected for the evaluation 
sample.* Thus, while the final district sample did not contain the entire 
evaluation universe, it did include all of the districts most fully meeting all 
of the selection criteria. 

Each district was defined as a stratum, and Qne school was randomly sampled from 
each stratum; the "best" matching sdhool was then selected as the other member - 
of the pair. One school of each pair was randomly assigned to the treatment 
condition, and the otherjschool to the control condition. No further sampling 
was performed on the Basic elementary evaluation universe, als the total of 47 
districts and 94 school? (47 pairs) fulfilled the initial design specifications 
of at least 25 pairs of schools. 

3. BASIC SECONDARY SAMPLE 

The Basic secondary evaluation universe consisted of all districts receiving 
Basic secondary grants in April or June 1973, having two or more pairable schools 
with Basic program activities designated for grades 10/ 11, and 12. A 'further 
design goal was that there should be at least 50 students in each of those grade 
levels in each of the participating schools. Twenty Basic secondary districts 
met these criteria, and all were selected. Thus, as in the Pilot program, the 
sample was identical to the evaluation unaverse*** 

i 

A pair of secondary schools was randomly selected from each,,pistrict, using 
the same procedure described for the elementary samples. Even though this 
resulted in fewer than the 25 pairs of schools desired for the Basic secondary 
sample, it was decided that sampling more than one pair of schools in a district 
would be too'' great a hardship on the district. One school of each pair was 
randomly assigned to the treatment condition and the other school to the control 
cionclition*. 

C. sKmpLING students within SCHOOLS 



student selection Was the second stage of the two-stage probability ssimpling 
procedure. Although the student-level sampling procedure was similar for all 
three evaluation samples, the concentrations of participating students 
constituting the evaluation universes varied widely across schools and across, 
evaluation samples. In roughly 38% of the Pilot and Basic elementary schools. 



There were some later ad3ustments to the Basic elementary sample to meet special 
conditions in certain districts. Final adjustments to this sample and*^ to the 
Pilot elementary and Basic secondary samples are described in Chapter III. 

As in the Pilot and Basic elementary samples, later sample adjustments were 
required in some Basic secondary districts to meet special conditions (see 
Chapter III) . 



/ 

1%, to 30% of the s,tudents were participating in ESAA/^>rograins , while in 25% of 
these schools, 31% to 60% of the students were invpived in ESAA activities. -In 
the remaining 38% of the Pilot and Basic elementary schools, between 61% and 
100% of the students were participating. 

The Basic secondary schools had a somewhat/<lif ferent. pattern than the elementary 
schools. A larger percentage of these sphools (48%) had between 61% and 100% 
of their students participating in- ESAik^'activities;' a smaller percentage of 
schools (28%) had only 1% to 30% of t^ieir students participating in ESAA. 
Approximately 25% of the Basic secondary achools had 31% to 60% of their students 
participating in,ESAA. 

Rosters of students participating in ESAA were requested, by grade, from treatment 
schools. The control school's were asked to provide rosters of students who 
would have received the Spone ESAA-funded services as the treatment school, if 
they had not been desigpated as control schools. Students with severe mental, 
physical, or linguistic handicaps were excluded from all rosters. 

Students' names wefe chosen by a random procedure within grade from the rosters. 
For. the third gr^de of the Pilot sample, 90 students, if available, were 
selected across' all classes within a school at , that grade level. The first 60 
of the 90 wej?^ included in the final sample if they were present and were tested 
during ^^al?.' achievement testing* The remaining students were included in the 
sample, yd the ordei: in which they were drawn, as needed to replace .absentees 
and to ireach the desired total of 60 . 

The same procedures* were used for Pilot fourth and fiftli grades, except that 
45 names were drawn'' for each grade (the desired final total being 30 per grade).* 
For the Basic samples, 90 names were drawn to obtain 60 students at each grade 
level. ^ 

In small schools or small ESAA programs, it was often difficult to reach the 
desired total of 60 or 30 studonts per grade. In these, cases, testing personnel 
asked school authorities, just prior to the test day, whether any now students 
mooting study criteria had arrived in school since the rosters had. been sent: to 
SpC. If so, names of these new students were added to the end of the previou>^ly 
prepared lists, in alphabetical order, and the students were drawn into the 
samples as needed. v / 



D. SUBSEQUENT SAMPLE REVISIONS 

As noted ab©ve, some additional adjustments were required in the samples to 

meet hardship -cases and other special conditions in some of the districts. In 

a few cases, for example, it was not possible to designate a control school in 
% 

*More Pilot students were sought at the third grade than at the fourth or fifth 
qrade, to allow for a longer period of attrition, i.e., up to three years. 
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a district r and two treatment schools were selected instead; such d^astricts 
were not included in the experimental (treatment-control) comparisons^ but 
were used- only for descriptive purposes. These final adjustments^/ and their 
effects on sample composition at the district and school levels, are discussed 
in Chapter III. Resulting student-level samples are described in/ Chapter IV. 



E . REPRESENTATIVENESS OF SAMPLES j 

Within the. total universe of ESAA Pilot and Basic grant awards constraints 
Were imposed to limit the districts and schools from which the evaluation 
samples would be drawn. These constraints (selection criteri^) , which 
reflected the priority goals of the evaluation, served to define three sample 
universes: Pilot elementary, Basic elementary, and Basic secondary. A 
particularly important aspect of thtj sample universes was the emphasis on 
selection of matched pairs of designated ESAA schools withi^n districts, so 
that schools could be randomly assigned to treatment and control conditions 
while holding district-level variables constant across eaoh school pair. 

Overall/ the three evaluation samples can be assumed highly generalizable to 
their corresponding evaluation universes, by virtue of tne procedures applied 
in selecting those samples. For example, at the di stride t level, the Pilot 
elementary sample included all of the award districts ^eeting the criteria 
that defined the primary Pilot evaluation universe (i^e., districts with Pilot 
programs awarded in April or June 1973, having two o^ more pairable schools 
designated for Pilot activities in at least two consecutive grade levels of 
interest, and with at least 25 participating students in each of .those grades). 
To augment this sample, several additional districts were selected that did not 
fully meet the criteria of the primary Pilot evaluation universe; these districts 
were used for descriptive purposes, but not for treatment^control comparisons. 
Similarly, the Basic secondary sample included all of the districts meeting 
the criteria of the evaluation universe, i.e., all districts receiving fiasic 
secondary grants in April or June 1973, having two or more pairable schpols 
with Basic program activities designated for grades 10, 11, and 12, and at 
least 50 participating students in each grade in each of the schools. 

Selection of sample Basic elementary districts was slightly more complex, but 
should still have produced a sample that was highly representative of the 
evaluation universe. The definition of this universe was identical to that 
for the Basic secondary univ*erse, but involved Basic programs at grades 3, 4, 
and 5 rather than at the secondciry level. The selected sample included all 
of the districts from those polled that fully met criteria for the evaluation 
universe. 

At the school level, it can probably be assume'ld that each sample (Pilot elementary, 
Basic elementary, and Basic secondary) was essentially representative of schools 
located in the selected districts and meeting the evaluation universe criteria. 
These school-level criteria included pairabiiity in terms of student SES, racial/ 
ethnic composition, and prior achievement/ and sufficient numbers of students 



IT-7 



24 



designated for participation at each grade level of interest. Within each 
sample group, the selected schools were a random' sample of school pairs in the 
corresponding evaluation universe. 

m 

Finally, it can be assumed that the student samples for all three sample groups 
were representative of eligible students in the evalxiation universes, since they 
were randomly drawn from designated ESAA participants at grade levels of interest 
in the selected schools. 

Although the discussion up to this point has focused on sample representativeness 
with respect to the evaluation universes, another issue of concern is whether 
the samples were||epresentative of the total universe of ESAA awards. Relatively 
little information is available orTthis subject, because of the lack of data on 
most characteristics of the award universe. Such sample-vs. -universe comparisons 
as can be made are presented in Chapters III and IV, which describe the sample 
districts, schools, and students. On a priori grounds, however, it was anticipated 
that the sample districts would represent an' oversample of large and medium-sized 
districts and an under sample of small districts. Such a trend would be a natural 
consequence of the intentional emphasis on matched pairs of schools, as such pairs 
would not be available in many of the smallest ^districts. 
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CHAPTER III 



DISTRICT, COMMUNITY, AND SCHOOfi CHARACTERISTICS 

This chapter is concerned with the demographic and organizational context in 
which ESAA prosirams operated during the initial year of program implementation. 
More specifically, descriptive information is presented on the regional and ^ 
state distributions of the ESAA award universe and the sample districts; 
relevant features of the final data analysis sample for Year I; deraographac 
and socioeconomic characteristics of the districts and schools conducting the 
sample ESAA programs; and the racial composition of the student bodies and 
staffs in the selected schools. 



A. REGIONAL AND STATE DISTRIBUTION OF AWARD UNIVERSE AND SAMPLE DISTRICTS 



1. REGIONAL DISTRIBU'?ION ^' 

Table III-l shows the regional distribution of both the ESAA award universe 
districts and the 93* evaluation sample districts. Of the 511* districts 
receiving ESAA awards, almost 70% were concentrated in two HEW Regions, while 
three regions had less than 5% of the total awards. Regions IV and VI (South-, 
east and South Central regions, respectively) received the largest numbers of 
grants, while Region I (New England) received less than 1% of the awards and 
Regions VII and X (Central Midwest and Pacific Northwest) received less than 
2% each. 

The jzegional distribution for the evaluation sample was very similar to that 
for the award universe, as indicated by a .97 correlation between the number 
of districts sampled from each region and the number of districts roceivinq 
awards in that region/ Almost 60% of the sample districts were located in 
Regions IV and VI, reflecting the heavy concentration of Basic and Pilot program 
grant awards in those areas. No sample districts were drawn from Region I 
(New England) or Region VIII (Western Mountain), which together received only 
4% of the total awards. 



2, STATE DISTRIBUTIONS 

Table III~2 presents the distributions of award districts and funds in the Pilot 
award universe and the Pilot sample. Thirty-five percent of the districts 
receiving Pilot awards were represented in the sample. The sample districts 



*This total figure counts districts that received both Pilot and Basic awards 
as two^ districts. 
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TABLE III-2, STATE DISTRIBUTION OF AWARDS IN PILOT UNIVERSE AND SAMPLE 



state 

• 


Number of 
Districts 


Total Dollars 
Awai;ded to 

Di Qt* T*! pt* in 

State 


Number of 
Districts 


Dollars Awarded 
tp Districts 

in Sample 


Percentage of 
Funds Awarded 

Represented' in 
Sample 
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Missouri 


1 


443,767 


1 


443,767 


100.0 


New Jersey 


4 


1,174,452 


3 


1,113,096 


94.7 


New Mexico 


6 


495,400 


2 


179,770 


36.3 


New York 


3 


5, 575, 188 




510,743 


9.'? 


Ohio , . , 


2 


921,781 


2 


921,781 


100.0 


Pennsylvania 


2 


. 1,578,260 


1 


1,456,510 ' 


92.3 


South Carolina 


8 


1,250,123 


2 


330,345 


26.4 


Texas 


11 


2,151,750 


3 


1501,540 


23.3 


Virginia 


7 


861,760 


4 


505,086 


58.6 


TOTAL for States 
Represented xn 
Sample 


81 


18,974,301 


34 


8,192,645 


43.2 


TOTAL for States 
Not ^Represented 
in Sample 


17 


2,361,460 


. 0 


0 


0.0 


TOTAL for All 
States 


98 


21,308,761 


34 


8,192,645 


38.4 
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* 

I 

received 38% of all Pilot funds and over 43%. of the Pilot funds in the sample 
states • In sever, of the 16 states represented in the sample, over 50% of the 
Pilot award dollars were accounted for by districts selected for the sample. , 

The correlation between numbers of sample ^districts in each state and numbers of 
districts in those states receiving ESAA awards was .54^ this is. a moderate 
relationship, but shows weaker re^Jresentativeness than that at th3 regional 
level. A still lower correlation (-20) was found between the amount of funding 
received by sample districts in each state and the total ESAA fundifig for all 
districts in that state. It should be noted, however, that the evaluation 
design did not pl^ce major emphasis t)n either regional or state representativeness 
of the sample. Rather, the sample design focused on the inclusion of matched 
pairs of treatment and control -schools; this requirement led to an oversampling 
of larger urban districts, and, at the state level, undoubtedly detracted from the 
overall representativeness of the sample with respect to numbers of funde(J* dis- 
tricts and> levels of funding. 

Table III-3 shows data on sample and award distribution by states for the Basic 
award universe and the combined Basic sample. Fourteen percent of the districts 
receiving awards were represented in the sample. Sample districts received 29% 
of all Basic funds and 32% of funds awarded in states represented in the sample. 
In nine of the 25 states, over 50% of the Basic award dollars^ were accounted for 
by .sample* districts. The correlation between the number of sample districts in 
each state and the number of districts in the state receiving awards was .56, 
indicating a moderate degree of association. The correlation between the amount 
of funding received by sample districts and that received by all districts in 
the sample states was .66, indicating a fair degree of relationship. \ 

r 

3* REGIQ^^AL DISTRIBUTION OF SAMPLE SCHOOLS 

The'regional distribution of sample c^chools, shown in Table III-4, is similar 
for all three evaluation samples. Regions IV and VI (Southeast and South Central)* 
had the largest numbers of sample schools, reflecting the concentration of ESAA- 
funded districts and schools in those regions. The percentages of sample schools 
that were located in Regions IV and' VI ranged from 51% for the Pilot sample to 
79% for the Basic Gocohdary sample. , The smallest number of sample schools was 
located in Region )C which had six Basic elementary schools, but no Pilot 
elementary or Basic secondary schools. , * 

4. SUMMARY OF GEOGRAPHIC REPRESENTATIVENESS OF SAMPLE 

As noted in Chapter II, the Pilot elonentary and Basic secondary evaluation 
samples included the total unive?rse of districts meeting the selection criteria, 
and the Basic elementary sample included most districts meeting those criteria. 
There seems little doubt, therefore, that all three evaluation samples were 
highly representative of their corresponding evaluation universes in all relevant 
dimensions including geographic, distributions . The present chapter examined the 
extent to which the evaluation samples were geographically representdtive of the 
ES/vA award universes. Although such representativeness was not a ma jor^ design 
goal of the evaluation, a very high correlation was found across the evaluation 
samples between the number of districts receiving ESAA awards in each region 
and the -number ot sample districts in that region. Geographic representativeness 

29 ' 
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TABLE 111-3^. • STATE DISTRIBUTION OF AWARDS IN BASIC UNIVERSE AND SAMPLE 



state 


Number of 

Award 
Districts 




Total Dollars 

Awarded to 
Districts in 
State 


Number of 

Sample 
Districts 


Dollars Awarded 
to Districts 
Represented 
in Sample 


Percentage of 
Funds Awarded 

WW O al«V^ wO 

Represented in 
Sample 


Alabama 


29 


5,582,978 


4 


1,650,-435 


29.6 ^ 


Arkansas 


' 30 


2,890,890 


2 


"Kll fi4h 

<J t t f 0*4V^ 




California 


15 


7,64S,595 


2 


739,631 


9 7 


^Di^trict of 
Columbia 


1 


1,962,418 


i 


1,962,418 


loo 0 


Florida 


12 


6,755/357 


,3 


1,621,636 


24,0 


Georgia 


28 


5,371,583 


1 


1,033,053 


19.2 


, Illinois- 


9 


3,^104,791 


2 


110, 288 


24.8 


Indiana 


f 1 


447,883 


1 


- 447,883 


100.0 ^ 


Kansas 


3 


614,575 


1 


45^,849 


74.5 


l^uisiana 


23 


4,240,180 


4 


947,220 


.22,3 


Maryland 


3 


1,892,966 


1 


1,019,471 


53.9 


Michigan 


2 


^90,000 


1 


459,000 


77,8 


Mississippi 


12 


3,335,474 


2 


1,878,799 


56.3 


Missouri 


3 


892,667 


1 , 


674,798 


75.6 


New Jersey 


9 


1,323,604 


2 


370,416 


28.0 


New Mexico 


9 


1,178,140 


2 


429,400 


36.. 4 


New York 


9 


11,716,609 


5 


3,0-16,719 


25.8 


North Carolina 


32 


8,265,258 


2 


616,882 


^7,5 


Ohio 


3 


695,765 


1 


\ 262,612 


37.7* 


Pennsylyania 


9 


2,868,951 


'3 


1,429,279 


49.8 


South Carolina 


• 26 - 


4,624,806 


2 


733,259 


15.8 


Tennessee 


9 


4,911,074 


4 


3y 075, 195 


62.6 


Texas 


50 


10,669,100 


7 


5,070,020 


47.5 


Virginia 


22 


5,449,571.^ 


2 : 


1,031,224 


18.9 


Washington 


4 


838,2JL0 


3 


801,920 


95.7 


TOTAL for States 
Represented in 
Sample 


353 


97,868,445 


59 


31,147,247 V 


31.8 


TOTAL for States 
Not Represented 
in Sample 


60 


8,405,239 
» 


0 


0 


0.0 


TOTAL for All 
States 


413 


106,273,684 


59 


31,147,247 


29.3 
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was considerably more modest at the state level, and when looking at ESAA 
dollars awarded rather than at number of district grants awarded. ^ 



B. DATA ANALYSIS SAMP LE FOR YEAR I . • 

A certain amount of shrinkage in the sample of schools Selected. for the 
ESAA evaluation occurred between the final sampling of districts and 
the pretest administration. In many cases, adjustments were made so that 
district-G coul(? remain in the sample, but in some instances, districts were 
dropped from 'the study. In addition, some attritipn in the study sample occurred 
between th^ Fall of 1973 and the Spring of 1974. All adjustments and changes to,, 
the original evaluation sample are discussed below, and the final Year I samples 
resulting from these adjustments are described. The sample modifications are not 
mentioned again in subsequent chapters, but analyses reported in' those chapters 
are based ^on the adjusted samples unless otherwise noted. ^ 

1. ADJUSTMENTS . 

The grea,t majority of selected districts agreed to participate as stipulated 
in the sampling design. However, some districts staged that their , partici- 
pation would create severe problems at the local. level. Generally, these latter 
districts requested one of tlie following: 

a) the selection of an alternative control school, either because the 
selected schopl had the greatest need in the district for the funds 
or because ESAA. activities were required undet a desegregation plan. . 

b) the exemption of the selected control school from that status, because 
having a control schoo^^ wQuld result in serious educational and/or 
political repercussions . 

These requests were handled in a variety of ways, depending upon the specifics 
of each request. ^ Wher^ hardships were claimed, resolutions were worked out: 
directly between USOE and the districts 4 In some cases it was necessary to 
select new pairs of schools; when this occurred, those pairs were randomly 
drawn and the treatment-coil Lrol conditions were randomly assigned- In other 
cases , treatment and control sections were Established within schools . These 
sections contained students who were randomly assigned to treatment and control 
classes. Finally, in some control schools, limited ESAA activities were allowed 
in specified grades outside the grades of interest in the evaluation. 

Because of the comjparatively small number of secondary schools in most districts, 
secondary control schools tended to create the greatest burden for districts. 
Some districts requested that no control schools bq employed, and it occasionally 
appeared that no control-type adjustment was reasonably possible; pairs of 
treatment schools were selected in these cases, but were not included in any 
treatment-control comparisons. . 

in addition to these adjustments, several districts made changes in their programs 
during the yea^r that caused tliem to fall outside the boundaries of the survey 
universe. For example, some districts withdrew from ESAA, and some districts 
reassigned funds to grade levels not in the study or to schools not selected for 
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participation. In these cases, the districts were 'dropped from the study. 
Also, two large urban districts in easjtefn pities did not release spring 
.questionnaire and posttest data unt:il it was too lat,^ to include them in 
the Year I analyses. One district had a large percentage of Black students 
and the other had the* largest Spanish-descent population of the districts 
originally included in tlxe evaluation sample. ^ . , 

^ 2. ATTRITION IN SAMPLE SIZE BETWEEN PRETEST POSTf EST ADMINISTRATION 

Exclusive of schools in the two large urban districts where posttest dataVwe'ire 
not available, the pretest sample consisted of 21,699 students in 180 school^; 
19,573 of these . students were posttested, for an overall attrition rate the . 
first year of only, 9.8%. 

As would be expected, the attrition rate for secondary students was much higher 
than for elementary students (17.7% for secondary students vs. 7,4% for ele-- 
mentary students).. Within the elementary level, the three grades had almost 
identical attrition rates. For the secondary students/ however,, the rate of 
attrition increased markedly in the twelfth grade. This may be in part explained 
by students' reaching an age where they were no longer subject to compulsory- 
education requirements. Overall, elementary students in Basic programs had a 
slightly higher attrition rate than those in Pilot programs (7.9% vs, 6.4%). , 

Among students whose -ethnicity could be established^ the highest attrition 
r^te in, the Pilot and Basic elementary samples occurred among white students 
(approximately .9%) . in secondary schools, Blr.cks efnd Whites had virtually 
identical attrition rates (17%) . There was no difference in attrition rate 
between treatment (ESAA-funded) schools and control (non-funded) schools for 
any* of the three evaluation samples, 

9 

In bpth elementary groups, a much higher than average attrition rate occurred 
in HEW Region VII (Centif-al Midwest); in\he Basic elementary prbgrams only, a 
similarly high rate occurred in Region IX (Western states) . Among Basic 
secondary schools, three of the four regions represented had quite similar 
attrition rates. Hov/ever, in Region II (New York and New Jersey) the rate was 
double that in the othqr regions. 

An important question was whether the shrinkage in sample size was selective 
in terms of , achievement level; that is, were the students who dropped out 
substantially hiJjher or 16wer in terms of their pretest achievement scores than ' 
.those! ^students who were still in school at the time of the posttest? On all 
measures, in alj, but one instance (Basic elementary Grade 5— -Reading Vocabulary), 
the pretest mecihs of the students who were not posttested were lower than the 
means for the ^tudents who were posttested. This difference in scores was greater 
at the seconda^ level than at the eleruentary levels presumably ^ec^use of the 
greater attrit^ion rate in high schools. For example, in the twelfth grade, 
where student attrition due to dropouts was particularly heavy, the difference 
in scores for pretest-only and pretest-posttest students was largest, 

The evidence seems quite clear that, overall, it was the poorer-achieving 
students who left school, either as . dropouts or as transfers. To this extent, 
any findings concerning student gains in achievement test scores may not be 
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entirely generalizable to the original (pretest) sample, which included 'a 
larger proportion of the slower students than the final posttest ^sample. 
No systematic.differences were found between ti^wdtment and control sqhools in 
attrition rates or in the pretest achievement levels of students not present 
for posttesting. . ' 

3. SCHOOLS IN FINAL (PRETEST-POSTTEST OVERLAP) EVALUATION SAMPLE 

The final Year One evaluation sample consisted of schools for which ccmplete 
pretest-posttest achievement data were collected. Table III-5 shows the 
distribution of treatment and control schools in the three sample groups on 
which all Year On^ analyses were performed. Overall/ the Pilot sample consisted 
of 18 treatment-control pairs and 29 unpaired treatment schools; the Basic 
elementary sample consisted of 38 pairs; and the Basic secondary sample con- 
sisted of ♦ 12 pairs and six unpaired treatment scliools. However, the distribution 
of paired treatment an<J control sctiools and unpaired treatment schools differed 
at each grade level. As the table indicates, thei^e were some school pairs and 
unpaired schools where only one or two grade levels Were represented. The last 
two columns of the table specify the number of paired' and unpaired schools v?ith 
student' participation at all three grade levels. 

School units were divided into separate schools (i.e., schools^o feeder- fed 
relationship where grade 3 is in one school and grades 4 and 5 are ih another)* • 
Also, treatment-control sections within schpols haVe been separated and treated 
as separate schools. . 

In the experimental analysesiTit was ajjpropriate to look only at the paired 
treatment and control schools in $acli sample.. The descriptive analyses in- 
cluded thq unpaired treatment* schools as-well. The composition of the sample 
used for each type of- analysis is specified where appropriate,' in later 
sections of this report, 

C. RACIAL/ETHNIC COMPOSITION OF STUDENTS AND STAFFS IN SAMPLE DISTRICTS 
AND SCHOOLS 

I 

1. SAMPI4E DISTRICTS 

District student body racial/ethnic ccxnposition, which was a major factor 
determining district Basic and Pilot award eligibility) is presented by HEW 
Regipn in Table III-6. These figures are based on district-level data collected 
by the U.S, Office of Civil Rights; racial/ethnic groups were categorized as 
"White (Anglo)/* "Black," "Spanish sur named ; " "Asian," or "American Indian," 
Blacks were by far the largest minority group, followed by Spanish-sumauned 
"Students. 

U.S. Department of HEW Regions showed varying degrees of - racial/ethnic 
heterogeneity. For example. Region X (Pacific Northwestern states) , which 
included oniy sample Basic award districts, had extremely light minority- 
group representation among students, with very littlje variation, dn the other 
hand, regions that included both, sample Pilot and [sample Basic award districts 

/ ' 
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showed more racial/ethnic diversity among districts.'^ Heavy concentrations 
of minority students were especially evident in districts in Regions III and 
IV (Mid-Atlantid and Southeastern), with an average of more than 50% Black 
student enrollment. Regions II and IX (Northeast arid Pacific Southwest) had * 
an average of at least 25% Spanish-surnamed students. 

Staff racial/ethnic composition estimates 'were obtained from EShh district 
sttperintehdents and included racial/ethnic breakdowns for teachers, admini- 
strators, and clerical staff. The' general pattern sliowed non-minority-group 
dominance among ESAA district staff. Regions III and IV (Mid-Atlantip and 
Southeast, respectively), however, reflected considerable racial heterogeneity 
in the staff, especially at the teacher level. In these* cases, where average 
Black student enrollment exceeded 50%, Black and White teachers tended to be 
almost evenly distributed. The South Central and Pacific Southwest Regions, 
(Regions VI and IX) , v/bich had subst^tial percentages of Spanish-sv^rnamed 
students, showed at least 10% Spanish-background representation on the average 
across all staff levels. 

2. SAMPLE SCHOOLS 

Table' III-7 presents the student racial/ethnic composition of sample schools 
in the three evaluation groups. This information is based on questionnaire 
responses of the school prii^cipals. In both Pilot and Basic student bodies. 
Blacks were by far the major minority group represented, followed by 
Spanish-background students. " * " 



TABLE III -7. STUDENT RACIAL/fiTHNIC GROUP COMPOSITION 
IN SAMPLE SCHOOLS 



Racial/Etlinic 
Group 


Pilot 
(Total N = 65) 


Basic Elementary 
(Total N = 76 


Basic Secondary 
(Total N = 30) 


%" * S.D. *N 


%" S.D. N 


% S.D. N 


White/Anglo 
Black 
Spanish- 
Background 
Other 


21.8 14.2 37 
65.1 29*^ 37 

11.6 ' 26.8 37 
1.4 4.5 37 ' 


55.7 26.3 62 
38.0 26.9 62 

,4.4 14.0 62 
2.1 7.7 62 


56.4 32.8 "26 
41.0 V 31.0 2^, 

0.5 - 1.2 26 
1.0* 2.1 25 



designates mean percentage. 

16' 



Pilot program objectives were focused on assisting children in minority- impacted 
schools, that is, schools exceeding 50% minority -group enrollment. Thus, it is 
not surprising that Pilot schools had heavier minority-student enrollment and 
greater minority representation among school staff than did Basic schools. In 
contrast to Pilot programs, Basic elementary and secondary sample schools were 
characterized by more racially -mixed student bodies. 

Racial/ethnic student body composition varied considerably among sample schools 
within the tliree evaluation groups. Table III-8 shows the frequency 
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distributions of the percentage of minority students represented in the 
student bodies of the sanple schools- A13 but one of the Pilot sample schools 
had 50% or irvore minority-group enrollment; Basic sample school^, varied more 
widely in the percentage of minority students, ' Altliough the Basic ESAJV grants 
were for districts that had desegregated 6r were in the process of desegregation 
during the 1974-75 school year/ several of the Basic elementary and secondary 
schools in the sample had student bodies that were either virtually- all npn- 
minority or predominantly minority. Thus, the mean percentages of each racial/ 
ethnic group specified in Table III-7 for the aggregated Basic sample may be 
somewhat misleading. As Pilot sample school-^ had consistently high percentages 
of jninority students, the mean percentages presented ia Table "III-7 for this 
samole are more in terpre table. 



TABLE III-8, PERCENTAGE OF MINORITY STUDENTS REPRESENTED 
IN SAMPLE SCHOOLS IN THE THREE EVALUATION GROUPS 





Number of 


Number of Basic 


I^umber of Basic . 




Pilot Sample 


Elementary 


Secondary 


Percentage of 


Schools 


Seunple School^ 


Sample Schools 


Minority Students 


(Total N=:65) 


(Total N=76) 


(Total N=30) 


0-10 


0 


0 


0 


11-20 


0 


9 ' ' 


4 


21-30 


0 


15 


2 


31-^40 


0 


13 


3 


41-50 


1 


7 


4 


51-60 


4 ' 


2 


1 


61-70 


8 


3 " 


1 


71-80 


9 


5 


1 


81-90 


8 


2 


1 


91-100 


7 


5 


3 V 


Number of Missing 








Cases 


28* 


15 


5 



Tcible III-9 shows staff racial/ethnic composition in samplt: £>ohools at teacher, 
administrator, and clerical levels. For all samples, but particularly for the 
Pilot Sample, the percentage of minority-group staff members was considerably 
lower "-than the percentage minority-group students 

The average percentages of minority students and staffs m paired treatment 
and control schools are shown for each evaluation group in Table III-IO, There 
were no statistically significant treatn\ent-control difierences in the percentages 
for any of the evaluation groups. At the student level, these results indicate 
success in obtaining treatment and control school pairs that were matched in 
minority-group representation. 
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TABLE III-9. STAFF RACJiAL/ETHNIC COMPQSITION 
IW SAMPLE SCHOOLS 



Racial /Etlinic 
Group 


Pilot 
(Total N=65) 


Basic Elementary 
(Total N=76) 


Basic Secondary 
(Total N=30) 






S.D. 


N 


% . 


S.D. 


N 


% 


S.D. 




TEACHERS • 






















White/Anglo 


51 


p 






fi7 ft 


^ J . o 


/ V 


' / *j . 4 


xo . / 




Black 


OO 


Q 

• o 




ou 


--0Q Q 




/o 








Spanish-Background 


C. 


p 




50 


** • o 




/ V 




n 7 




Other 

- 


0 


.4 


2.2 


49 


1.0 


2.9 


7^ 


0.8 


3.8 


28 


AnMTMTCTDZiT'nDC; 

AUMi N J. o Jl rCAi Ut<o 






















White/ Anglo 


43 


-P 


43.6 


49 


74.8 


38.7 


75 


64.2 


33.9 


28 


Black 


38 


.0 


43.9 


51 


18.6 


35.7 


76 


33.6 


33.8 


28 


Spanish-Background 


11 


.2 


29.1 


48 


2.0 


12.6 


75 


0 


0 


28 


Other 


1 


.0 


9.1 


48 


3.1 


' 11.2 


74 


2.2 


6.3 


28 


CI^IlCAL STAFF 






















White/Anglo 


48 


.8 


44.8 


51 


68.9 


37.9 


75 


71.6 


34.1 


28 


Black 


41 


.2 


45.1 


51 


, 22.3 


35.3 


76 


27.4 


34.2 


28 


Spanish-Background 


7 


.1 


,21.8 


50 


5.1' 


18.7 


75 




0 


28 


Other 


1 


.9 


13.7 


50 


3.4 


11.7 


75 


,0.9 


1.0 


28 



* _ 

"%" designates mean percentage. 
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TABLE IIItIO- PERCENTAGES OF MINORITY STUDENT? AND STAFF IN SAMPLE GROUPS 



/ 

/ 

Sample Group 


Treatment Schools 
in T-C Pairs , 


' 

Control Schools 
in T-C Pairs 


%* 


S,D, 


N 


% 




M 


A, Pilot E'v'aluation Sample 














' STUDENTS 


80.0 


18,0 


6 


73,2 


14.6 




N\jmber of Missing Cases 






12 








TEACHERS 


45.9 


21,0 


10 


49,8 


22.8 


10 


Nutnber of Missing Cases 






8 






8 


ADMINISTRATORS 


49.3 


44,1 


11 


73,3 


45.8 


11 


Number of Missing Cases • 






7 






7 , 


CLERKS 


55.3 


46,3 


11 


73,3 


45', 8 


11 ' 


Number of Missing- Cases^ 






7 






7 


B, Basic Elementary 






• 








Evaluation Sample 














STUDENTS 










^ O , 0 


O / 


Number of Missing Cases 






12 






1 


TEACHERS 


32,8 


21,8 


37 


31,8 


23,5 


37 " 


;i|Iumbex of Missing Cases 






1 






1 


ADMINISTRATORS 


27,3 


41,2 


36 


23,6 


37.7 


36 


Number of Missing Cases 






2 






2 


CLERKS 


31,2 


37,3 


36 


32,1 


39.8 


36 


Number of Missing Cases 






2 






2 


C. Basic Secondary 














Evaluation Sample 














• STUDENTS 


' 42,0 


35,5 


8 


44,0 


36.8 


8 


Number of Missing Cases 






4 






4 


TEACHERS 


24,6 


, 20,2 


10 


29,5 


21,^7 


10 


Ni^er of Missing Cases 






2 






2 


ADMINISTRATORS 


34,3 


38,2 


10 


39,8 


41,5 


10 


Number of Missing Cases 






2 






2 


CLERKS 


34,6 


36.5 


10 


34,1 


42,5 


10 


Number of Missing Cases 


\ 




2 






2 



*"%" designates mean percentage-. 
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3. CORRELATION BETWEEN PERCENTAGE OF MINORITY STUDENTS AND PERCENTAGE OP 
MINORITY. STAFF WITHIN SAMPLE SCHOOLS 



Given the discrepancy between petcentage of minority-group representation in 
student enrollment and on school staffs, particularly in the Pilot sample, it 
was of interest to determine how strongly the two minority-group imeasures 
correlated across schools. Correlation, coefficients (Pearson's r) were com- 
puted between the percentages of minority staff members at the different sample 
schools and the percentages of minority students in those bame schools. 

In, the Pilot evaluation sample, the percentage of minority teachers and the 
percentage of minority students were not significantly correlated (a = .05) 
ia'either treatment or control schools; one possible inference is that there 
was no personnel policy in the Pilot districts for attempting to achieve racial/ 
ethnic balance of teaching staff compos itiop with student body composition. 
However, in the treatment schools, percentage of minority students did correlate 
significantly with percentage of minority administrators and cle!rical staff. 

Sample Basic programs showed much greater correspondence between staff racial/- 
ethnic "Composition and student body racial/ethnic ccxnposition than the Pilot 
programs. The percentages of minority membership at each staff level were 
significantly correlated* (the r's range from 145 to .94) in both treatment and 
control schools at the elementary and secondary levels. In terms of overall 
regional- distributions, the Mid-Atlantic and Southeast regions, which had the 
highest concentrations of minority students, also had the largest percentages 
of minority staff members. 

4. SUMMARY " • > , ' ' 

At the district level, the HEW Regions showed considerable variability in 
stuiJent racial/ethnic composition. The Mid-Atlantic and Southeastern regions 
showed heavy concentrations of Black students (over 50%) , while concentrations 
qf Spanishrsurname students were evident* in the Northeast (26%) and Pacific 
Southwest regions (35%). Minority-group^ studen^is were least represented in 
the Pacific Northwest region, vheire Blacks. (15.*) were the largest minority 
groyp' represented, and where the largest ^concentrations of Asians (6%) was 
found compared to other regions. ' ^ ^ 

At the school , level, Pilot sample schools (by definition "minority impacted!') 
showed mbre than 50% minority-group enrollment, while Basic sample schools 
were more racially heterogeneous. Blacks were generally the major minority 
group represented, followed by Spanish-surname students. The percentage of 
Spanish-surname students reported in^ Basic secondary schools was extremely 
small. 

Tlic percentages of minority staff personnel' were consistently lower, district 
for dxstrict, then the corresponding percentages of minority student enroll- 
ment. Overall, there was abetter correspondence (higher correlation) between 
the student h d staff minority-percentage figures in Basic sample schools than 
in Pilot schools: "Districts in the Mid-Atlantic and Southeast regions, which 
had high concentrations of minority students, also tended to have racially 
balance<i teaching staffs. ^ " ^ , 
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D. COMMUNITY; DISTRICT, AND SCHOOL CHARACTERISTICS 



1. SIZE OF 'SURROUNDING COMMUNITIES ^ * . . 

One contextual dimension bf interest in characterizing sample ESAA programs is 
the size of the embedding communities. For summary purposes, communities with 
populations of 50,000 or greater (including suburbs) were classified as 
"mediiim/large/* while Communities of under 50,000 were considered "small." By 
this definition^ Basic sample schools were predominantly set .in medium/large 
communities (see Tabla III-ll); 66% of sample Basic elementary schools and 79% 
of sample Basic secondary schools operated in such communities. By contrast, 
the Pilot sample showed wider community-size distribution, with about even 
proportions of small and mediuin/large districts. 



TABLE III-ll. COMMUNITY-SIZE CONTEXT OF SAMPLE PROGRAMS 



size of Community 


*Pilot Sample 
(N=65) 
(Percentage of 
~ Total) 


Basic Eleinentary 
Sample (N=76) 
(Percentage of 
Total) 


Basic Secondary 
Sample (N=30) 
(Percentage of 
Total) 


Small 


53 


- 34 


,21 


Medium/Large \ 


. 47 


66 


79 


TOTAL N 


55 


76 


28 


Number of Missinq Cases 


10 


0 


2 



As mentioned earlier, large metropolitan areas in the Northeast area were not 
represented in tiiis evaluation, but many of the sample schools were drawn from* 
large cities and suburbs in other areas such as the South Central, Southeast, 
and Midwest regions* 



2. 



SIZE OF SAMPLE DISTRICTS AND SCHOOLS ' 

J * r . ' 

4II-I2 shows the distribution cf enrollment sizes Jn sample districts and 
District enrollment size, defined by the average daily attendance, 



Table 
schools''. 

varied widely within the samjLle, The average enrollment was approximately 
25,000 pupils, but the range was from slightly under 400 pupils to over 200,0(ro. 
The substantial proportion (43%) of relatively large districts (greater than 
10,000 enrollment) in the sample is especially no*:eworthy. Wide variability in 
enrollment is also evident at the school level, with values langing from about . 
50 pupils to over 2,800 pupils. However, over half of the sctools had enroll- 
ments in the range of 300 £0 700 pupils • ^ 



3. PERCENTAGte OF DISADVANTAGED STUDENTS IN DISTRICTS XnD SCHOOLS ^ • 

A measure assumed to be related to student body socioeconomic status (SE3) 
was the, proportion of disadvantaged students, as defined th§ Federal 
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TABLE III-I2. DISTRICT AVERAGE DAILY ATTENDANCE (ADA) 
AND TOTAL SCHOOL ENROLLMENT 



District 


School 




^ 0 r^r « 4 A n t 

c tequency 


Jrercenurtye 


bnroiimenu 


F|:equency 


Percentage 


401-1000 


2 


3 


51-100 


5 


3 


1001-1500 


2 


3^ 


101-200 


5 


3 


1501-2000 


6 


8 


201-300 


10 


6 


2001-2500 


2 


3 


301-400 


18 


11 


2501-3000 


3 


> 4 


401-500 


34 


20 


3001-3500 


2 


3 


501-600 


20 


12 


B501-4000 


1 


1 


601-700 


17 


10 


4001-4501 


1 


1 


fOl-800 


11 . 


7 


4501-5000 


2 


3 


801-900 


8 


5 


5001-6000 


1 


1 


901-1000 


7 


4 


6001-6500 


3 • 


4 


1001-1100 


8 


5 


6501-7000 


3 


4 


1101-1200 


3. 


2 


7001-8000 
8001-9000 


2 
7 


3 
10 


1201-1300 
1301-1400 j 


3 
0 


2 
0 


9001-10,000 


3 ^ 


4 


1401-1500 [ 1 


0.5 


, 10,061-15,000 


5 


7 


• 1501-1600 f 3 


2 


15,001-20,000 


3 


4 


1601-1700 


1 


0.5 


20.(301-30.000 


7 


10 


, 1701-1800 


6 


A 

V H 


30,001-40,000 


3 


4. 


1801-1900 


2" 


1 1 


40,001-50,000 


3 


4 


1901-2000' 


2 


1 


50., 001-60, 000 


3 


4 


2001-2100 


1 


0.5 


60,001-7^,000 
70,001-100,000 


3 


4 


2101-2200 


1 


0.5 


1 


1 


2201-2400>^ 


0 


0 


100,001-200,000 


3 




2401-2800 


1 


0.5 


200,000 + 


1 


1 


* 2800 + 


1 


0.5 


TOTAL 


72 


100 


TOTAL 


' 168 


100 


Mean 


25075.31 


Mean 


714,33 


S,D, 


39735.07' 


S.D, 


497,52 




< 


43 






i 
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req^uirements of the Elementary and Secondary Act (ESEA) , Title I, at the 
district and school, levels. Estimates of these proportions were obtained from 
sample district superintendents and school principals. It should *be noted 
that the results obtained from these two sources may not be directly comparable; 
district-level estimates of Title^-I-eligible students are defined solely in 
terras of economic need, while the school-lev^l estimates by the principal may 
tend to reflect educational need as*well. 



a. District-Level Distribution 

Table III-13 summarizes the regional variation in percentages of economically 
"disadvantaged" students in sample districts. Region III (Mid-Atlantic), 
Region IV (Southeast), Region vr (South Central)^ and Region IX (Pacific 
Southwest) at the elementary-school level showed relatively depressed SES levels, 
with average figures of about one- third or more disadvantaged students. These 
regions also had heavy minority-group representation in the student body. 

There were consistently larger proportions of disadvantaged students at the 
elementary level than at the secondary level; Region II (Metrqpoliban Northeast), 
Region III (Mid-Atlantic), and Region VI (South Central) showed at least 10% 
difference in the proportions , at the two school levels. These differences 
may reflect a tendency of the most severely disadvantaged students to drop 
out before or during high school. ^. 



b. School-Le ve 1 ' D i s tr i'bu t i on 

Tabl.e III-14 shows average percentages of "disadvantaged" students for Pilot 
and Basic treatment-control school pairs and for Pilot unpaired treatment 
schools. **Some caution is required in interpreting these figures, because * 
estimates of the percentages were not available for some school pairs, 
particularly in the Pilot elementary and Basic seconda3jy samples. 

Consistent with vhe "minority-group isolated" characterization of districts 
having Pilot programs. Pilot sample schools )nad a substantially higher .average 
percentage of educationally/economically disadvantaged students than Basic 
schools. Estimates in Pilot sample schools ranged from around 50% (paired 
treatment schools) to around 66% (paired control schools^); by comparison, 
Basic elementary sample school pairs were generally closer to about 39%, and 
Basic secondary sample school pairs showed still lower estimates of about 30%. 
There were no statistically significant ti^eatment-control differences between 
paired schools in any evaluation group* 

In general, the school-level estimates of percentage of disadvantaged Pilot and 
Basic. elementary sample students exceeded the district-level estimates. This 
finding appears to be cpnsistent with the contention, noted earlier, that school 
principals' included educational need as well as economic need in making their 
estimates of percentages of disadvantaged students. 
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•'4* COMMUHITY INVOLVEMENT AND. DISTRICT DECISION-MAKING 

a. Degree of Coinmunity 's Involvement 

Community participation in sample schools was defined in terms of the school 
principals' reports of (1) the presence of active parent organizations and the 
^ethnic composition of their officers, (2) the presence of an active Community 
Policy Committee, and (3') the percentage of paraprofessional/clerical staff 
drawn from community resources. The existence of an active parent organization 
and Community Policy Committee reflects school-community interactions and pro- 
vides an indication of the community's level of interest "in the educational 
process • The percentage of paraprofessional/clerical school staff that is drawn 
from the community certainly reflects community participation in the school, but 
may, in addition, be an indication of the socioeconomic level of the school 
community. 

Virtually ^11 Pilot and Basic sample schools- reported having an active PTA or 
other organized parent group. PTA leadership was predominantly mixed in racial 
composition in sample Pilot and Basic'^ schools . About 20% of the sample Pilo.t 
schools reported parent groups with all-minority officers. ' 

It is less straightforward to evaluate community invofvement by the presence of 
an active Community Policy Advisory Committee or similar group, since this is 
largely determined by the nature of the community's political structure. Never- 
theless, at the elementary-school 'level, sample Pilot schools evidenced more 
community involvement than sample Basic schools, with about 60% of the Pilot 
schools having an active community policy advisory group, in comparison to 43% 
of Basic elementary schools; this suggests a greater minority-group interest 
^jln decisions affecting elementary education in Pilot program communities . Over 
naif the Basic secondary sample schools reported having an active community 
advisory grou^; 

Finally, community participation can be exam?ned^ in terms of the percentage of 
paid professional and clerical support drawn from parents of children who attend 
school. In Pilot sample school pairs and unpaired treatment schools, about 23% 
of these paraprofessionals were parents. The porre spending values for sample 
Basic elementary pairs and sample Basic secondary pairs were 19% and 15%, 
respectively. 

There were no significant treatment-control differences in the Pilot or Basic 
sample on any of the indicators of community involvement described above • 

b. Degree of Centralization of Decision-Making 

Another community-involvement dimension of interest was the decentralization or 

centralization of decision-making in policy, issues. To explore this aspect of 

ESAA program settings, school ^decision-making was examined in the following 

three areas: classroom curriculum and media, budget allocations, and the hiring 

and firing of teachers. School principals were asked to indicate the parties 

included in decision-making in these areas from among the following: the district, 

the principal, individual teachers, the staff as a whole, parents, and students. 
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Greater decentralization was indicated by the inclusion of many parties in • 
decision-making, suggesting a more lateral organizational structure. Centrali- 
zation was indicated by higher-level decision-making, primarily by the district 
and principal 

The pattern of decision-making was fairly consistent across programs. There 
was greater decentralization ia decisions concerning classroom curriculurn and 
media; increasing centralization in administrative decisions concerning budget 
allocatibns; and almost total centralization in decjsiong concerning the hiring 
and firing of teachers. For example, in decisions concerning classroon\ curriculum 
and media, not only were educational staff involved but parents and students 
were also indicated by about one-fifth of Pilot and Basic -elementary sample 
schools; at the high-school level, moreover, over 50% of the sample schools 
indicated both parent and student involvement. Most Basic elementary (77%) and 
Basic secondary (90%) sample schools reported district involvement in decisions 
concerning classroom curriculum and media; by comparison, just over half the 
Pilot sample schools reported such district involvement. The staff as a whole 
was reported to be involved in curriculiim decisions by the majority of Pilot 
sample schools (80%) , ^ 

In decisions concerning the budget, a considerably smaller percentage of sample 
schools reported the participation of individual teachers, parents, and students. 
This is not surprising, since budgeting is traditionally an administrative issue. 
At the elementary schopl level, budget decisions primarily involved the staff as 
a vmole; at the secondary level, the district and the principal were the key 
parties indicated, ^ . 

Finally, in personnel decisions uonceiniog the hiring and firing of teachers, 
the district and principal were the major parties involved in decision-making , 
At least 80% of Pilot and^ Basic- sample schools indicated disti^ict participation. 
Principals also participated, as indicated by almost two-thirds of the Pilot 
sample schools and by a large majority of the Basic elementary (70%) and Basic 
secondary (83%) sample schools. 

There were no significant treatment-control differences in school decisions- 
making structure in either *^Pilot or Basic sample districts, - 
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CHAPTER IV 



STUDENT AND F.AMILY CHARACTERISTICS 



This chapter describes ^e family and student background characteristics, 
entrance achievement test scores, racial attitudes, attitudes toward 
school, "and educational expectations of sample students in the Pilot ele- 
mentary, Baltic elementary, and Basic secondary schools at the beginning of 
the first evaluatioi;! year (1973-1974) . Each of the three evaluation groups 
is discussed separately, and, where, appropriate, resi^lts are discussed by 
grade levels. In most rases, however/ results for all grade levels within 
a sample were very similar and are summarized "under one description. 

In this chapter and siibsequent chapters that include comparisons of the 
treatment and, control groups, differences between those groups are termed 
"significant" only when they reach the .05 level of statistical significance 
(i.e./ they would be expected to occur by chance no more than five times in 
a hundred)/. Differences that seem interpretable and of some particular 
interest, but that may not reach the .05 level of significance^ are 
characterized by tenths such as "apparent trends." 

A. FAMILY CHARACTERISTICS , - 

The description of the sample students* family background characteristics 
focuses on ^three major components: family size and whether the*student lived 
with his mother and/or father; parents' education and occupation; and two 
indicators of socioeconomic status (SES) . These descriptions were derived 
from student and teacher responses to several items included in the Spring 
1974 questionnaires. * 

l.^^JAMILY SIZE .AND COMPOSITION 

At the elementary level, family size for Basic students in all grades was 
typically between four and six members. • For Pilot sample students in all 
grades, "six" and "more than eight" people were the most frequently designated 
family sizes. At the secondary level, "four to five" people was' the most 
frequent response, followed by "six or seven." Most sample students (88%) in 
all grades and programs indicated that they lived wlrh their mothers. Fewer 
sample students, (64%-71%) lived with their fathers. 

Treatment-control differences for family size and for number of parents living 
at home were not significant at any grade level 4-n the three evaluation groups. 
However, there were significant differences in .these dimensions at all grade 
levels in the three groups between minority (Black, Asian American, American 
Indian, ' Spanish-background, and "Other") students and non-minority (White) 
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students. In general, minority students tended to have larger families (siXr 
seven, eight or more) than non-minority students, who usually had families of 
two, three, four, or five people* Non-minority students were more likely 
than minority students to live with their mothers and/or fathers. 



2. PARENTS* EDUCATION ^ AND OCCUPATION 

At the elementary level, the homeroom teacher provided estimates of the 
occupations and' levels of education of the students' parents. Because there - . 
was a high rate of nonresppnse (15%-50%) for these variables, the results must 
be interpreted with caution. At the secondary level, where the information was 
obtained from the students, nonresponse occurred less often (18%-26%). 

Teachers indicated that approximately two-fifths, of the mothers of Basic ele^ 
mentary students finished high school,. as did a third of the fathers. A 
greater percentage of fathers (15%- 18%) graduated from a foiir^year college 
than did mothers (10%). Responses for the Pilot students showed the same trends, 
except that more mo thers (13%) and fathers (16%) failed to finish grade school 
than did the mothers (8%) and fathers (9%) of the Basic elementary students. ^ 
Also, fewer parexj1bs<«^of the Pilot students attended and graduated from college 
(4%-6%) . * However- when those response percentages were calculated only for 
minority students in the two types 'of pri^grams, the differences in parents* 
education levels for the two programs virtually disappeared. 

Basic secondary students indicated that 43% of their ^mothers and 35% of their 
fathers graduated from higfi school but did not attend college.^ Thirty-five 
percent of the fathers and 26% of the mothers attended college. 

The most frequent occupation of mothers of Pilot, Basic elementary, and Basic 
secondary students was housework (46%) . Service work (16%) and clerical or 
office work (11%) were also common. Pilot mothers also were employed in 
-operative work (7%), which included^ jobs such as machine operator, assembler, 
welder, and taxi, truck, or bus driver. The most frequent occupations 'for 
fathers were operative w^tk (22%), labor (17%), crafts work (10%), professional 
work (11%) I and service work (8%). 

Differences in parents' education and occupation occurred primarily between 
minority and non-minority groups of sample students. Parents of minority stu- 
dents were less likely to have ..completed high school or college than parents of 
non-minority students. In addition, minority mothers were more likely to be 
employed in service or operative work, while non-minority mothers were- more 
likely to be housewiyes or clerical workers. Non-mj.nority fathers were fairly 
often employed as managers, professionals, or salesmen, while minority fathers 
were more often employed in operative work, service work, or labbr. 



3. HOUSEHOLD POSSESSIONS 

Two measures of socioeconomic status (SES) were calculated, based on different 
types of possessions present in the homes of the sample students. The first 
measurer- "SES/reading, " was related to the availability in the home of the 
following education-related materials: daily newspaper, dictionary, encyclopedia 
or other reference books, books of fiction and nonfiction, and magazines. The 
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second measure, "SES/luxury possessions," ^indicated the number of convenience 
or luxury items in the home. These included a color television, typewriter, 
dishwasher, two or more cars or trucks that operated, and an automatic clothes 
dryer.' Scoring on the two measures was based on a simple count of the items 
^within each group that the student indicated were present in his home. No 
statistically significant treatment-control differences were found within 
programs for either of the SES measures. 

Table IV-l presents the frequency distribution of the number of items in SES/ 
reading found in the homes of Basic elementary, Basic secondary, and Pilot 
students. The most frequent response for all groups was "five." There were 
statistically significant differences on SES/reading between minority and non- 
minority students within both Basic and Pilot programs. Minority students werq 
more likely to indicate that they had one, two, or three of the items, while - 
non-minority students were more likely to have all five items. Differences 
between the scores on SES/reading^ for minority Basic elementary and minority 
Pilot students at all grade levels" were also significant; means for the minority 
Basic elementary students were higher than those for the minority Pilot students. 

TABLE IV-1. AVAILABILI^^F EDUCATIONAL MAT5.RIALS . 
IN THE HOME (SES/READING) FOR BASIC JJEIiEMENTARY, 
BASIC SECONDARY, AND^PILOT STUDENTS 



SES/Reading 


Percentage of Students by Evaluation Sample 


Basic Elementary 


Basic Secondary 


Pilot 


0 


3 


1 * 


4 


1 


5 


2 


9 


2 


10 


A - 


14 


3 


16 


9 


21 


4 


24 


21 


26 


5 


42 


62 


• 27 



Table IV-2 shows the frequency distribution of the number of items in SE^ 
luxury possessions found in the homes of Basic elementary, Basic secqndary, and 
Pilot students. Indicating that they had four or five of the items in their 
homes were 35% of the Basic elementary students, 43% of the Basic secondary 
students, and.. 20% of the Pilot istudents. Again, all differences in SES/luxury , 
possessions between minority ^nd noa-minority groups in the three study groups * 
were statistically significant. Minority students were more likely to check 0, , 
I, or 2 items, while non-minority students werfe more lijkely to check 4 or 5 items. 
Diffe?:ences between the mean scores on SES/luxury possessions for the minority Basic 
elementary and minority Pilot students were significant at the third and fourth 
grades only. At thesd grade levels, the means for the minorit;y Basic elementary 
students were higher than those for the minority Pilot students • 

4. SUMMARY OF FA^1ILY CHARACTERISTICS , ^ 

In all areas examined, statistically significant differences occurred primarily 
between minority and non-minority students in either program type, rather than 
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TABLE IV-2. AVAILABILITY OF LUXURY AWD CONVENIENCE ITEMS^ 
IN THE HOME (SES/LUXURY POSSESSIONS) FOR BASIC ELEMENTARY,. 
BASIC SECONDARY, AND PILOT STUDENTS 



SES/Luxuiry 
Possessions 


Percentaqe of Students by Evaluation Sample 


Basic Elementary 


Basic Secondary 


Pilot 


0 


8 


7 


12 


1 


15 


'.2 


22 


2 


20 


17 


26 


3 


22 


21 


20 


4 


20 


22 


14 


5 


15 


21 , ; 


6 



between treatment and control students in any evaluation sample. Within either 
of the. programs/ ^minority stvdfents tended to come from larger families than 
non-minority students. More jtStudents lived with their mothers than with^ their - 
fathers, and non-minority students were more likely than minority students to 
live with their parents. Less than half of the parents (35%-40%) finished 
high school, while only 4% to .18% finished college; more non-minority than 
minority parents graduated from high school or college. Minority mothers and 
fathers tended to be Employed in operative or service work; non-minority 
fat^hers generally were employed as managers, professionals, or salesmen, and 
non-minority mothers were usually housewives or clerical workers. Basi^c ele- 
mentary minority students had higher levels on both SE^ measures than did Pilot 
minority students. 



B. STUDENT CHARACTERISTICS 

Sample student characteristics investigated in this section include sex, age, 
racial/etlinic group membership, academic needs (as indicated by Fall pretest 
achievement idlest scores), and student attitudes, beliefs, and expectations with 
regard to school and students of other racial/ethnic groups. In general, there 
were, virtually no statistically significant differences between the treatment and 
control groups of students at the beginning of the first ye^r of ESAA program 
implementation on any of these dimensions. This result indicates considercible 
success in matching treatment and control groups of sample students by random- 
assignment procedures. * 

- 

». _ 

X. SEX AND AGE 

^n each program for each grade, the scunple included about 50% females. The 
percentage of females ranged from 48% to 56%* Basic secondary grades 10 and 11 
were the least balanced in terms of percentages of female and male students, 
but even in those grades the females outnumbered males by only 12% or less. 

There was no great disparity in age among the students within a grade level in 
the three evaluation groups. The interquartile ranges for the median age showed 
at most a .7 year spread. 

« 
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2. RACE/ETHNIC GROUP 



Eligibility requirem^jnts for the Pilot and Basic programs imply jdifferences in 
school racial/ethnic composition between the programs. The Pilot program is 
designed to provide assistance for innovative compensatory- educatiofi projects 
in reading and math for children in minority-isolated schbols. In those schools 
it was not possible to achieve complete racial balance, andl one or more 
minority groups were usually dominant. The Basic program assists schools in 
reduping discrimination and minority -group isolation, as well as in developing 
programs to improve achievement in reading and mathematics; thus, in general, one 
might expect better racial/ethnic balance than in Pilot progj^am schools. 

Table IV-3 gives the racial/ethnic composition by grade level for all sample 
students in each of the two programs, as reported by teachers/ Each elementary 
grade level had a significantly higher pe^rcentage of Black and Spanish -background 
students in the Pilot sample than in the Basic sample, with a correspondingly 
greater percentage of White students in the Basic sample. Averaged across grade 
levels, the Pilot sample had 25% more Black students and 8% more Spanish-bacJ<- 
ground students than the Basic elementary sample, while the Basic elementary 
sample had 32% more White students than the Pilot sample. Within a program at 
the elementary level, the inter-grade variations in percentage of any partidular 
racial/ethnic group were small. ' ^ 



TABLE IV-3- RACIAL/ETHNIC COMPOSITION BY GRADE LEVEL OF THE 
PILOT, BASIC ELEMENTARY, AND BASIC SECONDARY SAMPLES 







Percentaqe of Students 


Grade Level 


Program 
Type 


' Black 


White 


Asian : 
American 


American 
Indian 


Spanish - 
Background 


Other 


3 


Pilot 
Basic ^ 


68-0 
44.1 


11.8 
45.0 


1.4 


2.1 
0,5 


17.6 
7.6 - 


0.4 
1.3 


4 


Pilot 
Basic 


68.6 
43.8 


13.9 
44.6 


0.3 
1.9 


1.9 
0.4 


15.6 
8.7 


0.^5 
0.^7' 


5 


Pilot 
Basic 


70.2 
44.7 


13.0 
44.8 


b.i 

|1.>6 


2.0 
0.5 


14.6 
7.3 


0.1 
1.2 


10 


Basic 


45.3 


53.5 


jo. 3 




0.6 


0.3, 


11 


Basic 


45.1 


53. e 




0.1 ' 


0.6 


0.4 


12 


Basic 


42.5 


56,4 






0.7 


0.3 



At the secondary level, the sample included only Basic students. There were " 
almost equal numbers of Blacks and \imitos at each grade level, and together these 
two groups made up almost 99% of the total Basic secondary sample. Thus, 
approximately 99^ of the minority students in this sample were Black. The very 
small percentage of Spanish-background students resulted from sampling constraints 
(described previously in Chapter II) that restricted the Basic secondary sample 
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to large ui^ban school districts, and from'fche fact that »the largest school 
district with extensive Spanish-background enrollments was not included in 
the final merged pretest-posttest sample . ^; I j: should be noted that as a ^ 
consequence, all secondary-level analyses in thijs report apply, for practical 
purposes, only to Black and White students. 

There were no statistically significant differences in racial/ethnic composition . 
between treatment and control schools in any of the threiS- evaluation samples* 
Thus one of the major goals in selecting matched pairs of schools, which was to 
equate overall racial/ethnic composition in the treatment and control samples*, 
was satisfied* 

3. STUDENT' ACADEMIC ^EDS * " ' 

All elementary and secondary students in the Year I Basic and Pilot samples were 
pretested in the Fall of 1973 to obtain measures of their reading and mathematics 
skills Tiear the beginning of the school year. , The achievement tests focused 
primarily on reading vocabulary, reading comprehension, math computation, math 
concepts, and matli problems (secondary level only) . Data were obtained in most 
sample schools in a six-week period from aroOnd the middle*- of October to the end 
of November. 

The evaluation design assumed that paired treatment and control schools would be , 
ciogely matched in their initial achievement levels. To test this assumption, 
statistical analyses based on an analysis-of-variance model were performed 
with the treatment-control pairs of jkchools in each evaluation group. The 5^ 
results showed no significant differences (a = .05) in pretest performance between 
treatment and control schools in either the Basic o^ Pilot samples in any of the 
reading or mathematics achievement areas. 

The pretest achievement scores for the Pilot elementary, Basic elementary, and 
Basic secondary students are described below. Although "debiased" scoring systems 
(i.e., scoring systems that excluded items determined to be potentially biased 
against minority students) had been developed for some of the tests*, it was found 
that differences in results between the full-scale scores and the debiased scores . 
were trivial and non-significant. Since the debiased scale scores were available 
for only some test forms and some grade levels, calculations of ^11 scores reported 
here were based on the full set of items originally included by the test ^ publisher . 
Summary tables specifying the grade equivalents and percentile rank for each sub- 
test are included for each evaluation group. More detailed tables indicating the 
numbet of items per subtest, means, standard deviations, range of scores, per- 
centile ranks, .and grade equivalents are included in the Appendix (Tables A-1 
through A-9) . 

a. Elementary Student Ach ieve ment in Pilot and Basic Programs 
(1) Pilot Elementary Students - " 

The percentile ranks and grade equiv^jilents for the mean pretest scores of the 
•Pilot and Basic elementary students (treatment and control combined) on the four 
subtests are presented in Table IV-4. Although the Pilot scores contained 

^ — - — - - — — - — — ■ I N I' ^ 

*For further information on the efforts to develop debiased scales, see Ozenne, D.G. 
Van Gelder, N.C., and Cohen, A.J. Emerggnpy School Aid* Act National Evaluation; 
Achievement Test Restandardizatxon ^ TM- 5236/006/00, System Development Corporation 
. November" 1, 1974. 
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moderate variability, all secures ^in the three grades represent depressed 
achievement test scores. Percentile finks for the various subtests ranged- in 
the high' teens and low twenties, indicating'^tnat approximately 80% of the 
students throughoXat the country were perfonning,.better than the average Pilot 
student. A similcir pattern can be ^en by comparing the 'pretest grade- 
equivalent scores in Table IV-4 withNihe scores thab^would nave^been expected 
of students with average achievement growth (based pn the publisher's national 
norms). Average students in grades 4, and 5 are expected to have grade 
equivalents of 3.1 -to 3.2 (third grtfde) , 4.1 to 4.2 (fourth grade), and 5.1 
to 5.2 (fifth grade) when tested at the start of the school year. The scores 
for the sample Pilot students were, consistently, w*ell below those average 
figures. . , . . 

Given the generally depressed level o^ test Scores^-' Pilot students' pret;^st 
performance in' all three grades was-]beVt in md*th computation and reading 
cpmprehension, and poorest in reading vocabulaxT^ and math concepts. None'pf 
the differences in pretest scores between the paired treatment and control 
schools, on any subtests, were statistically significant. 

It is clear that the ESAA Pilot ^^a\*ard process' resi^lted in selection of districts - 
and schools with academically needy students! as 4,naicated by their poor pretest 
performance on standardized achievement tests," and that the schools, by this same 
criterion, selected participant students with "^inarked 'academic »eeds. Thus, there 
is at least a tentative indicatioij^f a reasonable match between ESAA Pilot 
program re^sources (funding) and n^ed^.at the beginning of the%6hool year. How- 
ever, it is not possible on the basis of available evidence to demonstrate that 
the selection process necessarily led to participation of th e most needy districts ^ 
schools, or students. /> . 

*2) Basic Elementary Students ^ 

The percentile rank^and grade equivalents for the n\ean pr^test^g^cores of the 
Basic elementary students are, shown in 'fable IV-4, beside the corresponding* - , 
figures for the sample Pilot students. As in the Pilot sample, Basic elem^-* 
tary students had generally low achievement pretest scores", with percentiles 
for most subtests ranging from*tiie low twenties to the low thirties/ Scores 
for individuals covered a fairly broad range, indicating su^tantial' yar^- 
ability in entry ability despite the overall depression yf^roup means. Thd 
depression is confirmed by a comparison of grade-equivalent scores for the 
Basic elementary students (se.e Table/lVM) with the scores of average stude 
Based on national normative data, tjfe scores £or average students receiving 
the tests at the beginning of the sciiool year should correspond to grade ' ^ 
equivalents of 3.1 to 3.2 (third gvide) , 4.1 to 4.2 (fourth grade), and 5.1 to" 
5,.2 (fifth grade). The Basic students' pretext performance ran from half a 
grade equivalent to a full gradp^equivalent below those average figures. 

In the tliird grade, Basid elementary students performed best in reading compre- 
hension and vockbulary, and less well in math computation and math concepts.. 
For the fourth/and fifth grades, Basic students were wost proficient in mat^ 
computation ^d reading comprehension, and less proficient in reading ^vocabulary 
and math concepts. None of the differences in pretest achievement means between 
the paired treatment and control schools were significant. ^ 

Although the achievement levels of tlie Basic elementary students overall were 
hfglti^r than those of the Pilot students, this appears to be an artifact of the 




racial/ethnic ;coinposition of the two groups. Wlien only minority students in 
each program were considered, the achievement levels of the Pilot and Basic' 
elementary minority students were nearly identical. - ^ 

The ESAA Basic Gi-ant award process, like that for the Pilot program, clearly 
led to^the selection of districts and schools having students with severely 
depressed scores on standardized reading and matli achievement tests; furthermore, 
the students in those schools selected for Basic program participation had 
acute academic needs. Although these findings certainly suggest a good matching 
of Basic program funds with school and student needs, again there is no direct 
evidence as to whether the most needy of the districts/ schools, or students 
were selected for ESAA support. ' 



b. Secondary student Achievement 

Because the secondary students took the Iowa Silent Reading Test for reading 
achievement and the California Achievement Test for math achievement, it is 
highly questionable whether th^fe standard scores and siabsequent percentile 
ranks for the two tests should be compared to ascertain relative performance 
in the two achievement areas./ However, comparisons can be made between the 
grade levels within an achievement area. 

The percentile ranks and grade equivalents for the mean pretest achievement 
scores of the Basic secondary students are presented in Table IV-5. Grade 
equivalents for the reading subtests of the Iowa Silent Reading Test were not 
provided by ^e publisher." The achievement test scores were quite low in all 
areas for grades 10/ 11, and* 12, with means on the math subtests yielding 
•percentile ranks in the low twenties, and percentile ranks for the reading 
subtest ranging from 30 to 36. Approximately^ four-fifths of the students in 
the nation were performing better in math than these Basic secondary students, 
and two-thirds of the nation 's . students exceeded th6 sample students' per- 
formance in readi»»g. The Basic secbnd^ry students were less proficient in 
math computation than on other math subtests, and scored slightly higher in 
comprehension than in vocabulary. * 

Differences in pretest achievement means between the paired treatment and control 
schools were small and non-significant at the 10th and 11th grades. At the 12th 
grade, however, the pretest majrh achievement means of the control schools were 
significantly higher than the corresponding scores of the paired^ treatment 'schools 

In general, the results at the secondary level agree very closely with those found 
in the elementary samples, and show that districts, schools, and students selected 
for participation in Basic secondary programs had severe academic needs at the 
'beginning of the 1973-74 school year. Although it cannot be demonstrated that 
tHe selected students were needier than other students in the same districts and 
schools, if is clear that they were appropriate, targets for application of the 
ESAA funds. _ 
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TABLE IV-5. P3U3TEST ACHIEVEMENT LEVELS FOR .BASIC SECONDARY STUDENTS 



Test ' 


Percentile Ranks and Grade 
Equivalents for Mean Scores 


Grade 10 


Grade 11 


. Grade 12 


Percentile G.E. 


Percentile G.E . 


Percentile G.E. 


Iowa Silent 
Reading Test/ 
Level 2 

. Reading 
Vocabul^ary 


32 


33 


34 


. Reading 
Compr ehens ion 


33 


33 


36 


READING TOTAL 


30 


32 


34 


California 
' Achievement 
Test, Level 5 

Math Concepts 


24 8.1 


. 23 8.4 


24 8.6 


Math Computation 


^ 22 7.6 


-=*^ 

21 • 8.2 


2Q 8.4- 


Math Problems 


22 7.6 


20 8.1 " 


19 8.6 


MATH TOTAL 


23 ^ 7.9 


21 8.3 


'21 8.6 



c* Racial/Ethnic and Sex Differences in Academic Needs 



(1) Racial/Ethnic Differences 

Table IV-6 shows the percentile ranks and grade equivalents for the mecin pretext 
achievement scores in reading and math by racial/ethnic group within each of the 
three evaluation samples. V7ithin the Pilot and Basic elementary samples, data 
are not shown for some racial/ethnic groups that had only a small number of 
member studente. In the Basic secondary sample, only Black and \^ite students 
are represented. 

There were noticeable differences in pretest achievement level in total reading 
dnd total math among the various racial/ethnic groups within each evaluation 
sample. In general. Black and Spanish-background students had lower achievement 
test scores than White, Asian American, or American Indian students within each 
of the three samples. \ 
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(2) Sex Differences 

In both the Pilot and Basic elementary samples/ females had significantly higher 
pretest scores than males (a = .05) in vocabulary* reading comprehension, and 
reading tetal. Although females performed better than males 5-n tbtal math and 
math computation/ there were statfsticall^ significant differences between males 
,and females in the^math concepts area for only half the grade levels in the Pilot 
and Basic elementary sample schools. There were no consistent and statistically 
si.gnificant differences between males and females in the Basic secondary sample 
in any of. the reading areas, but male students in that sample made generally 
higher math, scores than female students. 

■ ■ . , • 

•i'§'?"-~. -Spitimary. of Student Academic Needs 

The most important implications of the consistently low pretest ,^cores in all 
evaluation samples are tl^^^at the ESAA award process was apparently' successful in 
focusing on districts and schools with academically needy^ students, and that the 
schools selected needy students to participate in the ESAA programs. Even though 
it cannot be directly' determined whether the selected schools and students were 
the most needy, this finding is still a very positive one, particularly in com- 
parison with findings from earlier studies of the Elementary and Secondary 
Education Act (ESBA) Title I.* Those earlier studies showed that high-expendi- 
ture districts with relatively dow percentages of needy children received a 
disproportionately high share of Title I funds? converseJLy, districts with high 
concentrations of needy children tended to receive disproportionately small 
shares of Tittle I funds. ^ 



4. {STUDENT ATTITUDES, BELIEFS, AND EXPECTATIONS 

! - « 

This section examines the racial attitudes., school preferences, attitudes toward 
schbol, and educational expectations of the sample students. Two sources of data 
were used. The Elementary and Secondary Questionnaires were administered in Fall 
1973 -Lo measure the baseline racial attitudes and any ^,changes in racial attitudes 
that might occur during the school year.** Additional items were also selected 
from the Elementary and Secondary Questionnaires adrninisteied in the Spring of 
1974. The racial/ethnic group identification item for the racial attitudes 
questionnaires included tlie following categories: Black/ \Vliite, Brown, and Other 
Because the intent was for students to self-select their racial/ethnic group, 
they^were not given explicit definitions of the different color designators. 



^Cf. V/argo,, Michael J., et al. ESEA Title I; A Reanalysis and Synthesis of 
Evaluation Data From Fiscal Year 1965 Through 1970 . TVmeri^^n Institutes for^' 
Research, Ralo Alto, California, March 1972." ; ^ 

**Because 6f school concern over some items in the questionnaires, the decision 
was made not to administer the questionna^ires at posttest time; hence only 
pretest data are available for the 1973-1974 school year. 

- *rf 
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Before reviewing the data, a brief note of caution should be sounded concerning., 
the difficulty in interpreting findings that cut across schools having' different 
racial/ethnic groups. \Tn a school having few representatives of the group in 
question, these attitudes will probably be based almost entirely on general 
stereotyped beliefs about that group;' in another school, where the group is 
abundantly represented, the attitude may reflect some combination of stereotyping 
and personal experience. Results summarized across both types of schools can 
thus be viewed only as overall trends not necessarily reflecting attitudes in 
schools having a particular racial/ethnic mix. . 

a* School PrefQjrences and Beliefs About Students pf.rOther Racial/Ethnic ■Groups 

Most sample students favoredca mixture of different racial/ethnic groups of 
students in their schools and classrooms. When students expressed a preference 
for one group, it was usually for schools or classrooms to have all members of * 
their own race or ethnic group. Such a preference was expressed more often 
by White, male, and/or secondary-level students. \ 

Sample students were asked if one racial/ethnic group caused the most trouble 
in school, or if one racial/ethnic group was more intelligent than the others. 
The prevailing opinion among Pilot and Basic elementary students in all grades 
was that no one racial/ethnic group created more diff iqulties in school than 
other groups, and thSt no specific racial/ethnic group was mpre intelligent 
than the others. However, a fairly large number of Black and Brown elementary 
students did indicate a belief that White students^ caused trouble in school. 

Responses of the Basic secondary students revealed differences in racial/ethnic 
group stereotypes. In general, each student rated his own. racial/ethnic group 
in the most positive light. White students^' were the most critical of students 
of other racial/ethnic groups. Roughly 58% of the students agreed that Black 
students were troublemakers; most of the students responding in this way were 
White, but a sizable number of Black students also selected this response. A 
substantially smaller number of students (both Black and White), felt that white 
students were troublemakers. 'Most secondary students (81.8% to 83.7%) responded 
that color did not have anything to do with intelligence. 



' t>. Attitudes Toward School and Educational Expectations 

Attitudes toward school were generally similar for the elementary and secondary 
levels. Most students felt that school was "Okay," but females and minority 
>^ students were more likely than males or non-minority students to feel that they 

liked school "a lot." Although most students in both the elementary and 
secondary samples felt that the rules in their s<rfiool were fair, non-minority 
students were more likely than minority students to express this attitude. In. 
both the Basic and Pilot elementary groups,, minority students were more likely 
than non-minority students to feel that they ^id not "belong." High-SES students 
in the Basic elementary grades were more likely to feel that they "belonged" in 
their school than were students of lower socioeconomic status. 

— MGSt sample students in both programs expressed a strong interest in getting 

good grades and in pursuing some type of post-high-school education. High SES, 
^ non-minority, and male (secondary-level only) students were particularly 
interestejJ in college-level work after finishing high -school. The social 

Gl 
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desirability of college or graduate school as an educational goal may liave had 
a strong influence in the pattern of their responses. 

r 

"College preparatory" (36%) and "general" (25%) were the most frequent types' 
of programs or curricula, in v;hich the Basic secondary students were enrolled. 
"Business" (18%), "vocational" (12%), and "work-study" (9%) were less frequently 
indicated. Males, non-minority students, and high SES students were likely to 
be enrolled in "college preparatory" or "general" programs, whereas females, 
minority students, an^'low-SES students were likely to be enrolled in "business" 
or "work-study" programs. 

C. SgMMARY OF RESULTS: SAMPLE STUDENT CHARACTERISTICS 

This chapter has described the family and student characteristics, entrance 
achievement test scores, racial attitudes, and attitudes toward school of sample 
students in the Pilot, Basic elementary, and Basic secondary schopls at the 
beginning of the 1973-1974 school year (the first year of ESAA program implemen- 
tation) . Some of the highlights of this chapter include the following-: 

^9 Students in both the Pilot and Basic programs had severe academic needs at 
the beginning of the school year, as evidenced by their consistently low pre- 
test scores on "the standardized reading and math achievement tes^ The 
students' performance level on these tests was generally below that of 70% to 
80% of the general population of students .in the grade levels of interest. 
The ESAA Grant Award process was evidently successful in targeting funds 
toward districts and schools with educationally disadvantaged students, and 
the schools designated needy students to participate in the JESAA program'. 
Further research is needed, however, to determine whether the ESAA award and 
student selection processes resulted in participation of the most needy 
districts, schools, and students. 

• In the Pilot programs, which by definition focused on minority-isolated schools, 
approximately 70% of the sample students were Black, 15% were of Spanish 
background, and 13% were White. Basic schoors had smaller percentages qf 
minority students (approximately 52% at the elementary level and 45% at the 
secondary level) , and correspondingly better balance in proportions of minority 
and non-minority students. 

• At each grade level, sample students were quite homogeneous with respect to age 
and were almost evenly divided between males and females. 

• Sample students tended to come from large families, and to have parents with 
a high-school education or less, although 4% to 18% of the parents (depending 
on the evaluation sample involved) -had completed college. Non-minority 
parents had, on the average, more professional or technical jobs than 
minority parents; they also had higher levels of socioeconomic status (SES) as 
indicated by reading-related materi-als and luxury items in the heme. At the 
olomentary level, families of Basic minority students had higher SES levels 
than families of Pilot minority students* 

• Most students in both Pilot and Basic samples expressed no deep-seated racial 
prejudices. A large percentage of the students favored schools and classrooms 
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with a mixture of different racial/ethnic groups. Expressed feelings 
toward school were also generally positive, with most students indicating 
an interest in getting good grades and in pursuing some type of post- 
secondary education. 
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CHAPTER V 



RESOURCE ALLOCATIONS: FUNDS 



This chapter deals with th^ allocation of funds among and within schools . 
Within-school allocations to different school activities and resources give 
one possible indication of program goals and priorities, while between-school 
allocations provide the basis for treatment-control funding comparison^ 
representing the only experimental variable in the ESAA evaluation. The 
evaluation model, as noted earlier, required the random assignment of one 
school from a matched pair of schools in each saitple district to the treatment 
(ESAA-funded) condition. The other school was then assigned to the control 
(non-ESAA) condition,^ The implicit assumption was that, except for the ESAA 
funds, the treatment and control schools in the district would be more or less 
equally funded.* ' One major purpose of the analyses reported in this chapter 
was tp assess the validity of this assumption. In particular, it was important 
to determine whether there was evidence that districts assigned other funds to 
the control schools to "compensate" for their not receiving ESAA money. To the 
extent that this happened, the experimental variable would be weakened, .and the 
chances of finding a significant difference in achievement between treatment and 
control schools would be correspondingly jpeduced. 

The chapter begins with a review of the methodology used in these analyses. 
Results of the analyses are then presented, beginning with a description of 
the levels of funding that sample districts and schools received from various 
souifbes, and following with a discussion of analyses comparing supplemental and 
total funding in treatment and control schools. Further analyses examine 
patterns of spending within paired treatment and control schools in order to 
examine possible differences in program focus. 



A. METHODOLOGY * i 



1. DATA COLLECTION 

A questionnaire administered to district business managers requested information 
for school years 1973-74 ^ (estimated) and 1972-73 (actual) on amount and 
spending patterns of compensatory funds; these funds were to include not only 
ESAA/ but also Title I, Title III, I'itle IV, ESAP, and special state funding. 
At the. district level, information was requested on average daily attendance 
(ADA) , available amounts of compensatory or remedial funding, and average 



*It should be noted that, by law, neither SDC nor USOE could direct districts 
to allocate non-ESAA funds 'in accordance with this assumption. Control of 
otlier federal program funds., such as KSEA Title I, was still at the discretion 
of the district, provided that the requirements of those other programs were met. 
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per-pupil expenditure excluding remedial or compensatory funding. Descriptive data 
on grade enrollment, on amount and source of compensatory funding, and on 
allocation pattern of funding by grade and by soujrce were obtained from the | 
district for each sample school. Follow-up telephone calls to the business 
managers made it possible to obtain a 90^ response to the questionnaires. 

The following definitions were used for the funding analysis: 

• Regular expenditures — A school incurs these expenditures in the course 
of running its regular school program, normally supported by state 
and/or local funds. In most districts^ the official district position, 
sometimes reflecting state regulations, is that regular per-pupil 
expenditures are constant across schools in the district. This was ' • 
accepted as a working assumption for purposes of the present analyses, 
although there is informal evidence that some districts deviate from 
this pattern in their actual expenditures.* 

• Supplemental expenditures — A school ihcurs these expenditures whei> it 
supplies remedial or compensatory services to low-achieving or 
disadvantaged;$phildren. Consequently, these are costs over and above 
regular expei^ditures and are financed primarily by state or federal' 
funds, including ESAA. 

• Total expenditures — Total expenditures are the sum of ;i:egular and , 
supplemental expenditures / 

. ^/ I 

2. COMPUTATION OF PER-PUPIL EXPENDITURES / j 

Because of wide variability in district size, it was important to utilize 'per- 
pupil rather than total expenditures. The calculation method employed is, 
^escribed below. • 

; 

k 

a. Regular Per-Pupil Expenditures I 

; 

Regular expenditures were reported directly in the questionnaire under t^e head- 
ing of "per-pupil expenditures" and were calculated by dividing the district* s 
current expense for education by the average daily attendance (ADA) . 



*A possibly extreme example of these within-district variations is a rt*cent 
study of a large metropolitan school district, which indicated that t^e regular 
per-pupil base varied firom $492 to $993 across Schools in the district. To 
obtain completely realistic data on individual school costs, it may be 
necessary to work "within the system," as was done in the study above; such 
an approach would be excessively time-consuming and expensive for a national 
sample such as that in ESAA evaluation. Nevertheless, the point should be clear 
that all cost data in this report are at best an approximation of actual dollar 
allocations. 
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b. Supplemental Per-Pupil Expenditures 

At the district level, the problem was' how to estimate per-pupil supplemental 
expenditures when no information was available on which programs (other than 
ESAA) were being applied to specific students, A-fter reviewing a number of 
alternative procedures, it was decided to use the best available proxy for the 
number of low-achieving or disadvantaged children in the district. Information 
had been obtained on the estimated percentage of children meeting Title I require- 
ments;* this figure was' selected and applied to the ADA** to arrive at a single 
estimate. The assumption here is that all remedial funds were applied to the 
same children, .Consequently, per-pupil expenditures will be overestimated. (or 
underestimated) if Title I requirements were less' (or more) restrictive than those 
of other funds, ^'^f* Hovvjever/ since information obtained informally from business 
managers indicated that approximately 70% of the pupilo receiving compensatory 
assistance got assistance from multiple funding sources, the assumption used 
here in calculating per-pupil expenditures may be an acceptable approximation. 

At the school level, information was obtained on the number of participants jn 
each compensatory brogram, but again not on the extent of overlap among students 
served by the^ fundsy The use of the average per-pupil grant size was rejected 
as a possible solution, as it was known that overlap exists in the students 
served by the different grants. This method could also produce estimates 
larger than the actual school enrollment. Therefore, it was decided to use 
the total school enro^llment as the divisor to ,oh>tain per-ptipil estimates. 
This approach seemed more justifiable at the school level than at the district 
level, since the sample schools typically had large proportions of students- 
in compensatory programs, whereas many^districts contained schools with large 
proportibns of non-par ^cipa ting students. In particular, the division of 
total supplemental funds by total School enrollment appears quite reasonable 
for programs such as Title I and certain special state funds., since the sample 
schools usually listed the entire student body as participants in those programs. 
The procedure will tend to underestimate per-pupil supplemental expenditures in 
the case of programs having smaller numbers of parcicipants, but the degree of 
underestimation should be similar for the matched pair of treatment and control 
schools that were used in tlie most important analyses. Since those paired 
schools were matched on socioeconomic status, prior achievement, and rcicial 
composition, it was reasonable to expect that they would have similar numbers 
of disadvantaged students,' In short, for the analyses reported here, the actual 



*Source was the District Superintendent Questionnaire B, Question 15. The 
value used was a weighted average of the percentage of the total enrollment 
meeting Title I requirements at the elementary and secondary levels in the 
district; the weights were the district enrollment at the respective levels, 

**0f the districts in the sample, 63% reported that 30% or less of the total 
district enrollment met Title I requirements, (However, most sample schools 
within those districts reported most or all of th^eir students to be partici- 
pc;iting in Title I,) 

***A potential source of error is in the use of the estimated proportion itself. 
As a multiplicative factor, its error rate influences several other computations. 
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per-pupil dollar amounts cannot be considered precise, but the relative treat- 
ment-control differences in funding should be meaningful. 



Clearly, tl\e procedure described is only one** of many different estimation models 
that could be used. In the absence of precise information about which specific 
^students received what ^id from which funding sources,- any method of calculating 
per-pupil supplemental expenditures is potentially subject to sizable errors 
of e^t'imatidh* Although the averaging procedures used for the first-year data 
wer^felt to be as good as any available, a continuing exploration of other / 
methoSs will be conducted for future reports. 

c. Total Per-Pupil Expenditur/s 

Total per-pupil expenditure ^/wer^ obtained by simply siamming ajegular and 
supplemental per-pupil exp^hditures. 

3. STANDARDIZATION 

/' 

In examining funding allocations, it was important to obtain standardized dollar 
figures that were adjiisted for local and regional differences in cost of living, 
so as to maximize comparability of data across districts, A consumer price 
index is published by the U.S. Department of Commerce* for 23 Standard xMetro- 
politan Statistical Areas (SMSAs) , but the most recent index data available 
were for 1972. Further, the index applied only to SMSAs, and did not include 
all the states in the sample or the small cities in tl-ie study. Therefore, a 
new index was constructed by dividing a particular district's minimum' teacher 
salary by the average minimum salary within the sample. This approach reflected 
variations between cities in the same state, and was based on the single largest 
resource expenditure of most sample schools. (Staff salaries represent: approxi- 
mately three-fourths of the total supplemental expenditures in reading and math.) 
The price indexes thus constructed ranged from .80 to 1.29. This reflected 
considerable variability^ in salaries, with beginning teachers in some districts 
earning more than half cjgain as much as teachers in lower-paying districts. 
Adjustments based on the prj-ce indexes should therefore have helped to reduce 
spurious variability in the expenditure data due to cost-of-living and other 
Ideal pricing. factors, i 



B. RESULTS 



The purpose of tliis section is to describe the general financial setting in which 
tlie sample schools operate. Results reported at the district level include 
standardized total, regular, and supplemental per-pupil expenditures. 



'•'Bureau of Census, Statistical Abstract, 1972. 
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a* Tqtal/ Regular; and Supplemental Per-Pupil Expenditures 

Standardized total per-pupil expenditure^ in the sample districts ranged from 
$601 to ^2,927 for the regular school year, with an average expenditure of 
$l/343. Tabl^ V-1 illus,trates this variation, showing that a majority of 
districts (51%) spent less than $1,200 a year, but it was not uncommon to * 
have expenditures higher than $1,800. The mean value of standardized regular 
per-pupil expenditures across all districts in the ESAA evaluation sample was 
$882, and 62% of the districts spent less than $900 per student,* 



TABLE V-1. STANDARDIZED TOTAL AND SUPPLEt4ENTAL PER-PUPIL EXPENDITURES 

FOR SAMPLE DISTRICTS, 1973-74 / 



Total Per-Pupil 
Expenditures 


Percentage 
Frequency / Total 


$ 600-$ 900 
$ 901-$1,200 
$1/201-51,500 
$1,S01-$1,800 
$1,800+ 


8 11 
29 40 
14 19 

9 13 
12 17' 


TOTAL 


72 100 


Statistics: Range = $601-$2,927 
Mean = $1,343 
Standard Deviation = $494 


Supplemental Per- 
Pupil Expenditures 


Percentage 
Frequency Total 


Under $100 
$100-$300 
$301-$50O 
$501-$70O 

$70i-$yoo 

Over $900 


1 . -1 - 
29 40 
22 31 
10 . 14 

1 1 

9 13 


Statistics: Range . = $75-$l,986 
Mean = $460 
Standard Deviation = $360 



Standardized supplemental per-PUPil expenditure— that is, funds from all sources 
other thian regular district sources—ranged from $75 to $1,986, with a mean of 
$460 (see Table v-1)- Seventy-two percent of the districts spent less than $500 
per pupil in supplementary monies • 
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b. Regional Variation in Per-Pupil Expenditures 

As ^howji in Table most of the districts in tjie ESAA evaluation sample were 

ia Regions IV (Southeast) and VI (South Central), the regions containing the 
most 'ESAA awardees* Regions II (Metropolitan Northeast) / V (North Mxlfiwest) , and 
VII (Central Midwest) appeared to have relativelj?^ higher total expenditures per 
pupil; this 'may have been because thio^e regions had the smallest proportions of 
students meeting Title I requirements (which suggests that tlie students were of 
higher SES) and also because they had large proportions of schools in urban^ 
areasy whichtyp jally have higher expenditures than rural, areas. Table V-2 
also shQws iT^pw the regular per-pupil expenditures were distributed adross the 
HEW regions. ' Regions IV and VI, which contained more than 50% of the,^skmple 
d4.'S,trdw;jts , again showed a concentration in the lower end of the expenditures 
scalfe; the Metropolitan Northeast, with seven district had the highest per- 
capita expenditure. The regional variation in supplemental funding followed 
thea.^sptine general pa-ttern as tliat for regular expenditua^es. ^ ' 

/ i 

^. SCHCX>L-IiEVEL FUNDING: DESCRIPTIVE ANALYSES i-. 

Paralleling the district-level descriptions, a series of analyses was performed 
on tlie funding characteristics of the schools in each 6f the three evaluation 
samples. The results of these analyses are contained in Tables V-3 tlirough V-5. 
A sumnary of the total per-pupil expenditures in standardized, dollars is presented 
in Table V-3, It is immediately apparent that the Pilot elementary schools had 
the highest total per-pupil expenditure, with Basic elementary and Basic 
'secondary schools expenjding smaller amounts. It should be. noted that the inter- 
quartile ranges of the three distributions are very similar, indicating a fair 
degree -of overlap. 

When these school distributions are compared with the one reported in Table V-1 
for the districts, it is apparent that the sample schools had lower per-pupil 
expenditures than the districts from which they were drawn. This does not mean' 
that the sample schools had. lower funding levels than other schools p.n the 
sample districts, but simply reflects the fact that administrative costs contained 
in the district calculations were not included in the school -level calculations. 

Table V-4 shows tlie per-pupil supplemental funding in standardized dollars. 
Again, the means and entire distributions were lower than the district counter- 
part reported in -Table V-1. Also, it is clear tiat the Pilot schools expended 
more supplemental funds tlian the ^asic ^elementary or Basic secondary schools. 
Noticeeible is the large difference in supplemental funding between the elementary 
and secondary levels, with the Basic secondary fur^ding level averaging less than 
a third of either of the elementary funding levels. However, it sfiould be noted 
that the secondary schools usually reported much larger numbers of student 
participants; with a larger denominator used in calculating the per-pjapil 
expenditures, the resulting estimates, * almost by definition, had to be smaller 
than those at the elementary level, 

A series of analyses, based on only jthe treatmeVit schools, was conducted to 
determine the nature and magnitude of ESAA funding. Table .V-S presents a 
descriptive summary of ESAA funding for all the treatment schools within each 
of the tlirce evaluation samples. The Pilot schools, on tlie average, expended 
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more ESAA money than did either of the other two groups. ESAA per-piapil/' 
expenditures were far lower at the secondary level than in elementary schools, 
again reflecting primarily the larger number of participating students reported 
at the secondary level, 'it may be noted in Table V-5 that one school reported, 
zero ESAA funding despite the fact, verified a telephone call to the 
district, that the school in question.. was designated as a treatment school. 
The cause of this inconsistency has not yet been determined. 

Another question of considerable interest relates to the use made by treatment • 
schools of the ESAA money. Was , it distributed for distinctly different 
applications than other supplemental funds, or treated more as part .of . a single 
supplemental pool? Data pertinent to this question are presented in Table V-6, 
which shows the division of money into reading, mathematics, and "other" 
activities. Expenditures in the "other" category consist of the sum of the 
expenditures for counseling and guidahCe services, new cun^^-culum development, 
intercultural relations programs, career education, community activities, 
administrative costs, and health, services. ' 

The data presented in Table V-6 indicate that, at the elementary level, a 
larger percentage of ESAA money than of non-ESAA supplemental money was focused 
on fehe "other" category, while the non-ESAA funding was more heavily concen- 
trated on reading. Little supplemental monfey seems to have been targeted to 
the mathematics area. .At the secondary level, by contrast, a greater percentage 
of ESAA mpney than of non-ESM supplemental money was spent in the reading area^». 
Once a^^n/ little ESAA money was spent on mathematics^ while roughly comparable 
percentages of ESAA and non-ESAA funds were expended in the "other" category. * 

A fitial series of analyses also examined the allocation of ESAA funds to * 
treatment schools; these analyses focused on the relationships* between numb,er 
of ESAA dollars awarded and characteristics of the school. Wo significant 
associations were found between the amount of ESAA funding and the r^ular * 
district-based funding or between ESAA funding and non-ESAA supplemental^ 
funding. At the secondary level, poorer , schools tended to receive larger 
afnounts of ESAA funding, indicating that these funds helped to ameliorate 
funding differences between, treatment schools. The amounts of ESAA funds were 
in'significantly but positively related to other supplemental funds received by 
treatment schools. 

Correlations between ESAA funding and initial levels of achievemeYit indicate 
that, at the secondary level, schools with the greatest' academic need (as 
indicated by the depressed levels of initial achievement in' reading and math) 
received the largest amount of support (statistically significant at p<.05). 
Relati0nships for Pilot and Basic elementary programs were non-significant. 

The relationships between ESAA funding and school demographic characteristics 
again indicate intejcesting patterns at the secondary level, but non-significant 
results at the elementary level. High schools witlj low socioeconomic levels 
received significantly higher levels of ESAA funding. There was a tendency 
for schools' with larger percentages of minority students to receive higher 



*Cor relational analyses were used for the results repprted here. 
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levels of funding, but this relationship was not statistically significant. 

All elementary-level relationships were near zero, with the exception of a 

low positive relationship between Pilot ESAA funds and the socioeconomic 

levels in those schools, 
« 

Certain conclusions can be drawn from the above discussion of school-level 
funding. First, local Pilot programs reportedly focused approximately two- 
thirds of their funds on activities directly related to the improvement of 
basic skills, and the remaining third on supportive activities such as 
individual and group guidance and counseling of students, new curriculum 
development, community activities, and administrative costs. Further explora-* 
tion will be performed in subsequent evaluation years to determine the degree 
of relevance of those supportive activities to skills improvement. In Basic 
elementary and secondary schools there appears to have been a reasonable 
allocation of jfunds across the three major application areas (reading, 
-mathematics, and supportive activities including desegregation-related 
activities), in light of the fact that Basic programs, under the Act, are 
intended to pursue desegregation-related goals as well as^Msic skills., 
improvement. • ' * 

The second major conclusion from the school-level funding anajDyses relates to 
the allocation patterns for ESAA and other supplemental funds. It seems clear 
that at le^st some of the ESAA money was distributed by treatment schools in 
a substantially different pattern than were non-ESAA supplemental funds. 

3. FUNDING AI^ALYSPS OF TREATMENT-CONTROL PAIRS 

« 

Because of the special interest in comparing fxinding levels and expenditure 
patterns in treatment and control schools, additional school-level analyses 
were restricted to a subset of sample schools consisting of treatment and 
control s<ihools in matched pairs. The subsample included 17 Pilot elementary 
school pairs, 33 Basic elementary pairs, and 12 Basic secondary pairs. 



a. Frequency Distributions of Funding 

Since regular per-pupil fxinds wete assumed to be constant across schools^ in a 
district/ treatment-control differences in supplemental funding were equivalent 
to differences in total ,f landing. The variable employed in the school-level 
analyses was the per-pupil supplemental funditji^g; i*e, , the total supplemental 
funds were divided by^the total humber of students enrolled in the school. 
The per-pupil expendituire was utilized in an effort to reduce the effect of 
any difference in size between tlie control and treatment schools. 

In Table V-7 the frequency and percentage distribution, mean, interquartile 
range, and standard deviation of total per-pupil expenditures are shown for the 
treatment and control schools in each program. The results indicate that the 
treatment schools received, on the average, from $29 (Pilot elementary) to $43 
(Basic elementary) more total 'per-pupil funding than the control schools. 
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Table V-8 presents the* same type of information for per-pupil supplemental 
fvinding. The results here mirror the pattern found in ^the previous tabid/ 
since regular district funds were assumed to be a constant for treatmentJcontrol 
pairs. It is immediately apparent that Pilot schools, whetlier treatment or 
control, received more per-pupil supplepiental funding than Basic schools. Of 
further note is the low level of funding for the secoridary schools where, 
because of larger reported numbers of participating students, "the per-student 
figures were less than a third as large as those for treatment pupils in the 
elementary samples. 

b. Treatment-Control Funding Breakdowns by Region 

Analyses were performed to determine geographic breakdown^of . the total and 
total supplemental funding levels for treatment and control schools. As 
expected on the basis of using matched pairs of schools, there was little 
difference between the treatment and control schools' mean fvinding levels 
within a geographic region. Differences across regions reflected the pattern 
previously discussed at the district level. 



c. Comparative Funding in Treatment and Co^itrol Pairs 

Matched-pair t-ratios were calculated to determine whether the funding 
differences in treatment and*control schools were statistically significant. 
The statistics were computed for total supplemental per-pupil funding and for 
non-ESAA supplemental funding. Each of these was further broken down j.nto the 
component funding areas of reading, mathematics, and "other/' with separate 
analyses being conducted for staff and non-staff expenditxires. 

For the Pilot sample, no statistically significant treatment-rcontrdl differences 
were fdund irj? any area of total supplemental dollars. This finding was at 
least partially explained by tHe analyses of non-ESAA funding. The control 
group had higher non-ESAA supplementary expenditures than the treatment group 
in every funding categoiry except non-staff expenditures in 'reading. While 
the differences were not statistically significant, they had the effect of 
reducing the treatment-control differences in total supplementary per-pupil 
allocations to a non-significant level. 

m 

In the Basic elementary sample, the treatment group has significantly higher 
total per-pupil funding except in the "other" category. No significant 
differences existed between the two groups in non-ESAA supplemental funding. 

In the Basic secondary sample there were significant treatment-control 
differences ('favoring the treatment group) in three areas: reading 'expenditures, 
total supplemental expenditures, and "other expenditures." There were no 
significant treatment-control^if ferences in non-ESAA supplemental allocations.. 

An additional analysis was conducted in an attempt to gain more insight into 
the magnitude of the treatment-control differences in total supplemental funding. 
Table V-9 presents the results of this analysis. A frequency distribution of 
the percentage differences m funding between the trea.tment and control school 
pairs is presented foif each evaluation sample. The percentage estimate for 
each treatment-control pair was determined by compulsing the difference betwer^n 
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per-pupil supplemental funding for the two schools, and dividing that difference 
)Dy the total per-pupil funding of the control school. 

•As can be seen in the table, over 46% of the Pilot sample pair?, 61% of the 
Basic elementary ^irs , and 75% of the Basic secondary' pairs were within .5% of 
zero difference. Furthermore, some of the differences actually favored the 
control schools. Differences of larger than 5% in favor of the treatment 
school were found in only 30% of the Pilot pairs, 36%. of the Basic elementary 
pairs, and 25% of the Basic secondary pairs. 

In summary, only in the Basic samples did treatment schools exceed controls 
in total Supplemental funding to a degree that was statistically significant. 
Should the funding differences for either the Pilot or Basic samples be con- 
sidered educationally meaningful? Unfortunately, this question is largely 
judgmental, and therefore cannot be resolved in any unequivocal manner. The 
largest mean difference occurred in the Basic elementary sample for total 
supplemental funding; here the treatment schools spent on "the average $46 more 
per pupil than did the control schools. This amount could buy several more 
textbooks for each student, but could not cover the cost of a full-time aide 
for the class. Readers must decide for themselves whether it is reasonable 
to expect such an incremental expenditure to produce significant differences 
In student outcomes. * » 

If previous studies are any indication, the ways in which the funds are applied- 
may- be at least as inportant as the absolute amount of the dollar differential 
The results of the comparative analyses discussed earlier are germane to the 
question of program focus; they examined the treatment-control funding differences 
fo*^ various application' areas within reading, mathematics., and "ather" activities 
Within the Basic elementary sample, there were statistically significant 
differences favoring treatment schools in total per-pupil supplementary funding 
for all areas of reading and mathematics; in fi.e Basic secondary sample, the 
differences were significant for all reading 'areas and for "other" activities - 
These^ analyses -give some indication of relative funding emphasis, but a clearer 
picture may be obtained by examining the^ percentages of total supplementary 
funds spent on different areas by treatment and control schools. This topic 
is discussed in the following section. 

4. ALLOCATIONS OF SUPPLEMENTAL FUNDING WITHIN SCHOOL PROGRAMs" 

Considering first the overall allocation of supplemental funds to reading, matli, 
and other activities, there were relatively small differences in the focus of 
treatment and control school programs at the eJewSitary level, but somewhat 
greater differences at the secondary" level. /Table v-10 presents the percentage 
allocations to each of these areas of focus for the total supplemental money in 
th,e paired treatment and control school?. • Of particular interest are the 
percentages focused on reading and mathematics activities, siijce these are 
activities presumably closely related to outcome measures used during the first 
year. Within the Pilot sample, control schools tended to spend a slightly 
larger part of their supplemental money on reading activities than did th^ir 
treatment counterparts. Neither treatment nor control schools spent much of 
the supplemental money on mathematics activities. 

Basic elementary, schools, a relatively high percentage of supplemental funds 
was expended for reading .activities, and smaller amounts were focused on 
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mathematics. Here the only differential allocation slightly favored the treat- 
ment schools in the mathematics area. Again, the difference was small and 
would probably not lead one tg^ infer a meaningful difference ^n program focus. 

Basic secondary schools hafl quite a different pattern of allocation, with "other" 
activities receiving approximately the same funding ^emphasis as the academic 
areas. Treatment schools placed more focus on reading than did the control 
schools, whereas the control schools emphasized "other" activities tnore heavily. 

5. SUMMARY ^ , 

• Sample Pilot schools allocated approximately two-thirds of their funds 
for activities directly related to basic skills improvement, and the 
othQr third for supportive activities such as -counseling and guidance,, 
curriculum development, associated administrative costs, and * community 
activities. ' -if • 

• Basic elementary and Basic secondary schools appear to have allocated thelir . 
ESAA funds in accordance with legislated program goals, since f^r the Basic 
pro.gram, the legislatrion includes desegregati^on-r^lated objectives as well • 
as improvement in basic skills. Local programs, on the average, committed 
sizable portions of their ESAA funds to activities addressing each of those 
major objectives. 

• There were significant treatmei'it-control differences in total supplemental 
funding for two of the three samples. In the Basic^ elementary sample, treat- 
ment schools jiad significantly larger total per-pupil expenditures than their 
paired control schools; these differences were produced by greater treatment- 
group expenditures in reading ^and math activities. However, the differences 

in total funding exceeded 5% in only 36% of the "pairs. In the Basic ' ^ 

secondary sample,, treatment schools again had significantly larger total (p6r- ^ 
pupil expenditures, because of higher allocations in reading 'anti "other" 
activities. In only 26% of the pairs was this funding difference greater than 
5%. There were nonsignificant treatment-control differences in total per-pupil 
funding across the Pilot sample. ^ < 

*» 

• For all sample districts (Basic and^ Pilot combined), the .average total per- 
pupil expenditure (standardized across districts) was $1,343. The average 
total supplemental per-pupil expenditure was $460. 

• The average total per-puj>il expenditure for all sample Pilot schools was $1,031. 
The Basic elementary sample had a mean value of $994 and the B^sic secondary 
sample an average of $879. The per-pupil supplemental expenditiuros varied 
from $238 for Pilot schools to $120 for Basic elemeptary schools and $32 

for Basio secondary schopls. 

• In Pilot treatment schools, the average KSAA expenditure was $232. In Basic 
elementary schools it was $174, while the Basic secondary samj>le had an 
average ESAA expenditure of $60 per pupil. ^ 
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The relative percentages of total per-pupil. supplemental money allocated 
to reading, inathQmatics, and "other" activities were quite similar for 
treatment and control schools tne elementaryT^fe^tgl^ At the secondary 
level, treatment schools spent a larger percentage than^control schools 
on reading, and a smaller percentage than controls on "either" activities • 
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CHAPTER VI 



RESOURCE ALLOCATIONS: SERVICES 



This chapter examines, the translation of program dollars into the provision of 
services to sainpre students in Pilot and Basic sample schools* It describes 
those services with particular emphasis on activities related to ESAA objectives 
and addresses questions such as the following: What kinds of facilities, staffs 
and ina.ter'ials were used in providing services to sample students? What were 
the characteristics of the reading and math instructional programs? Of int;er- ^ 
cultural programs? Finally , how were identifred student academic needs correlated 
with the receipt of services in ESAA programs? 

The data reported in this chapter are necessarily limited* in scope and depth. 
They were gathered by means of questionnaires, which cannot provide detailed 
information* on the quality of tlie instructional interactions, or on the tone 
or climate of the classroom and school environment. Moreover, the school pro- 
gram descriptions reported here are based- on responses by school staff members 
and administrators, and may not fully or precisely reflect actual occurrences 
in the classroons and schools. The picture presented in this chapter, using , 
data from the program questionnaires, will at a later 'date be supplemented by 
a document describing the results of m-depth, on-site observations of a subset " 
of [the sample sites. , 

One goal of the ESAA program, particularly in Pilot projects, was to provide 
compensatory education to students with identified academic ""deficiencies stem- 
ming from segregation and minority isolation. But ESAA is rarely the only com- 
pensatory program in the school; often, an ESAA-participating school also provides 
compensatory education funded under ESEA Title I and othei? Federal or state 
programs. These services are frequently extremely similar in nature, regardless 
of the funding source, and school staff members are often unable to specify exactly 
what activities are supported by what programs • Thus, any meaningful picture of 
the compensatory learning environment of students in the sample schools — 'controls 
aSj well as treatment schools — must include data on ^he full array of ccxnpensatory 
activitie's in those schools. This ^section of the report is intended to provide 
such data. In virtually all insftances, the instrulnents used to collect these 
data were administered in control and treatment schools, and did not specifically 
*distingu sh ESAA-funded compensatory activities from ndn-ESAA act^ities. 

The first section of this chapter describes the physical, .staff, and material, 
resources characterizing the students' school enyironment in sample programs. 
The next section provides a description of instructional program^, including 
such characteristics as program expbsure^ instructional objectives and approaches, 
classroom grouping characteristics, and lesson structure. Descriptive profiles 
are provided for sample Pilot and Basic programs in regular and remedial reading^ 
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regular and r^edial math, and inter cultural programs • The final portion of ,the 
chapter concerns the issue of whetfier ESAA Services were appropriately targeted . 
towards the nbeds of studehts'identif ied as having .45pecif ic academic deficiencies* 

Tables in this chapter display distributioAs for tifeatment and control schools , 
in matched pairs as well'^as distributionS^or unpaired treatHiMt schools* In 
general, the correlated t:-test was used to test for treatment-^ntrol differences 
at the school level, / * . * 

A, . RESOURCES AVAIIABLE IN SAMPLE SCHOOLS * ' . " * ^ ' - • 

!• PHYSICAL RESOURCES ' , ' 

Physical resources provided for sainple\tudents were examined in terms of the^ 
availability of central and spec^lized facilities , as /^^gepor ted by school princi- . 
pals. Central facilities , as def3,ned here, included' centyali-zed^ librarl'es; 
auditoriums; cafeterias; gymnasiums^ rooms housing var;Lou*s c;ombinat±ons bt'^ \ 
cafeterias, gymnasi\ans, and auaitoriums; kitchens; and athletic fields* Specialized 
facilities included resource centers such, as jCentral media centers, classroom . 
libraries, learning laboratories, instructional material's jproductfi^n centers, 
televisiqp, production studios, and teacher reference centers* 

Tnere were no significant differences between paired treatment and control schools 
in the availability of central or specialized facilities 'in PilDt and Basic s^ple 
groups* There were also no differenc'fe^ between Pilot and Ba^ic elementary sample*; 
schools, but secondary scTiools tended to exceed both elementary samples in central 
and specialized facilities, ' - 

Central media cen^^rs and classroom libraries were found in -the majority of bath * 
Pilot and Basic sample schools at the elementary level* About half of the'piiot 
and Basic elementary sample schools, moreover, reported having learning' labora- 
tories and teacher Jreference centers. More specialized facilities were offered 
at the secondary level, presumably to meet ^augmented student curriculum needs. 
For example, 76% of sample Basic secondary schools had instructidnal ma't^rials^ 
production centers, in comparison to 55%. of 4:he Pilot elementary schools, and, 64%^ 
of the Basic elementary sample scho61s. Central 'media ^enter^, classroom libraries, 
and learning 'laboratories "were al:so available in a n:aioritY of^ the Basie secondary 
schools. • ^ ^ c. ^ - ^ ^ ' ' 

2. STAFF RESOURCES * ^ r \ 

a. Number of Students Per Teacher * , •, . " 

' ^ * 

The number of students per teacher provided scxne indication of the distribution \ 
of staff ♦ resources among students in a school. Sample school principals were 
asked to estimate. the average total number of students per teacher for each grade 
level in their schools. Overall., there were no major differences across grade 
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levels within san^le groups. The^s^Mnmary data ^snowed that, within treatmejit- 
control pairs at the elementary scnpol level, the number of students per teacher 
was about 30 in Pilot Vainple schoolk, 27 in Basic elementary sampl'e schools, and 
•25 in Basic ^secondary sample schot>3:s. There were no significant treatment-control 
differences in overall student-teacher ratios. However, as noted in Section B 
of thi'^chapter, there were some significant treatment-control differences in 
cla^s^^^ze in reading and math instructional programs • 



b. Teacher Education 

The educational background of sample program teachers was examined in terms of 
the highest college degree earned. Among elementary program teachers, the most 
common response was the bachelor's dqgree. Fourteen to twenty percent of Pilot 
teachers and about 26% of Basic elementary teachers reported having the master's 
(degree, reflecting' slightly greater professional skills among sample Basic 
elementary .teachers • In Basic secondary schools, teacher education was alrfiost 
evenly divided betwe€fn the bachelor's and master's degree leyels-w No significant, 
treatment-control differences were found in any ot the evaluation samples. 



Teacher Inservice Training^ 



Inservice training to improve the teaching of reading or math, or to iielp 
teachers work with low achievers and racially mixed grc^vJ^s , was treated for 
this evaluation as a professional staff -development component of compensa- 
tory programs* Such training is also an indicator of how funding resources 
were allocated in the school. . 
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Teachers in the "ktudy. s^ple (treatment asid control)^ were asked to indicate. the 
areas in which they had received inservice training within the previojas 12 

, months frolfi among the following: reading, math, intergroup relations, -and'^ 
irfs€ruction of the disad\/antaged. At the elementary-school -l^vel , inservice f 
training activity was most heavily focused on reaffiing* In both Pilot and Basic 
samples, inservice training participation was highest in reading instruc\:ion , 
with about 50% of the teachers participating'. About one-third of the sample *^ 
teachers reported having* taken inservice training in math and* in intergroup 
relations. Instruction of "the^'disadvantaged hfid the lowest rate of , inservice 
training participation. There were no significant trea-toent-contrbl differences 
in the Pilot sample, but a significantly highejf proportion of teachers in Basic 
elementary treatment school's (52%) than in the corresponding control schools ^, 

* «,f87%) reported inservice training in reading. * • . ^ 

Compared with sample eletnentary schools, sample Basic secondary schools showed 
less inservice training participation in general, but greater emphasis t)n in- 
•service training related to intergroup relations. The proportion of secondary- 
level treatment school teachers (36%) who participated in this area of Anser- 
vice training was significantly^ higher than that of cQptrol school teachers 
u (2J%) . Instruction of the disadvantaged drew the next highest overall r^te of 
inservice training participation {]^5%) . • ' ' ' ^ ■ 
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d. Specialized Support Pisrsonnel and Teacher Aides 

Table VI-1 gives information on certain aspects of staif composition within - 
programs. For each sample group, the table shows the percentage of total staff 
size accounted for by specialized support personnel (i.e., remedial reading 
teachers, remedial math teachers, speech therapists, and audiovisual specialists). 
No significant treatment-control differences were evident in the Pilot or Basi.c 
evaluation groups. 

At the elementary school level, remedial reading teachers were represented most 
commonly among specialized support personnel, accounting on the ^average for 11% 
of the total staff in both sample Pilot schools and Basic elementary schools. 
Remedi'&l math teachers, speech therapists, and ^audiovisual specialists each 
represented/ on the average, 5% or less of the total staff in both Pilot and 
Basic programs. At the secondary school level, the representation of specialized 
support personnel was considerably smaller; remedial reading teachers and com- • 
pensatory math teachers each accounted, on the average-, for about 2% of the 
total staff. 

The practice of having , teacher aj-des in the classroon has become a point of 
interest in elementary education. In this study, teachers were asked to indicate, 
wtiether aides were available in their regular and remedial reading and math 
classes. Overall, teacher aides for both regular and remedial programs weref 
, present niore frequently in Pilot schools than in Basic schools. In Pilot regular 
reaSin^programs , there was a significant difference between the percentage of 
tre§,tment school teachers (61%) and control school teachers (37%) reporting one 
or more aides in class. Similarly, in Basic elementary regular reading programs, 
treatment school teachers reported the presence of teacher aides significantly 
more often than teachers in the paired control schools (42% compared with 25%)r. 

In regular math programs, both Pilot and Basic, treatment fechool teachers reported 
the presence of aides significantly more often than did the control school teacher 
Specifically, the percentages of treatment school teacffers reporting the use of 
aides were 52% (Pilot) and 36% (Basic elementary) , while the corresponding 
percentages of conta^ol "school teachers were 31% and 24%. 



3. MATERIALS USED IN^CLASS - 

Regular and remedial program teachers were asked to indicate the frequency of use 
of various types of materials, ^ach instructional resource type was ranked 
according .to Srequency of use by the teacher. 

Instructional JC.esources used in reading programs were first grouped according to 
whether they wp're printed or audiovisual. Printed resources included textbooks, 
other iDpbks and printed^ materials', newspapers and magazines and other periodicals, 
and teacher-prepared materials. Aud'iovidSfel resources included motion pictures 
and/or filmstrips, slides and transparencies, tape recordings and records, video 
or television tapes, and games. The»*types of instructional resources used •^in 
math programs were basic texts, drill-and-j^^tice materials, *and* audiovisual 
materials.*^ ; ( 



87 

VI-4 



TABLE SPECIALIZED SUPPORT STAFF COMPOSITION^N SAMPLE SCHOOLS 

A. PiJoC Evaluation Sample 



4 

Staff Type 


Treajtmeat 
Schools in 
T-C Pairs 
(N = 18) 


Control 

Schools in 
T-C Pairs 
(N ,= 18) 


Unpaired 
Treatment 
Schools 
(N = 29) 


% S.D. 


? .S.D. 


r S.D. 


Remedial Reading Teachers 
Remedial^ Math Teachers 
Speech Tl^erapists ^ 
Audio-Visual Specialists 


10.0 9.5' 
2.1 3.4 
2.5^ 3.2 

0** ^ 0 


11.2 9.8 
4.3 6.8 
3.5 3.9 

:> 3.3 8.2 


12.0 8.3 
6.4 7.9 
1.1. 1.7 
0 0 


Number of Missing Cashes 


6 


1 


^ 3 



B. Basic Elementary Evaluation Sample 



staff Type 


•Treatment 
Schools, ir. 
^T-C Pair's 
. (N = 38) 


Control 
Schools in 
T-C Pairs 
(N 38) 


%" S.D. 


? > ' S.D. 


Remedial Reading Teachers 
Remedial Ma^h Teachers 
Speech Therapists , 
Audio-Visual Specialists 


JLl.l 8.3^ 
5.2 7.7 . 
4.6 4.4 

.1.0 2.0 


10.5 ^ 9.6 
1.3 ' 4.0 
4.0 5.2 
1.7 4.7 . 


Number of MiiSsincr Cases 


0 


0 


* • ^ ; 
C. Basic -Secondary Evaluation Sconple 

f * 


• 

Staff Type 


Treatment 
Schools in 
T-C Pairs 
(H = 12) 


Control 
Schools in 
T-C Pairs 
(H = 12) 


%" S.D. 


S.D. 


Remedial Reading Teachers 
Remedial Math Teachers 
Speech Therapists 
Audio-Visual Specialists 


2.3 X.8 ^ 
1.9 1.9 
1.0 1.0 
1.2 1.6 


2.2 2.0 

1.4 2.7 
1.0 1.1' 

1.5 1,6 


Number of Missinq Cases . 


2 


^^ 0 
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Summary reading-resource indices were generated by averaging the ranks of the 
items comprising either printed or /udiovisual instructional resource sets. 
For math programs, single-item indicators were examined for the four math 
instructional resource types specified above. No significant treatment-control 
differences were evident in the use of either reading or 'math materials. / 

In regular and remedial reading programs, the pattern across both Pilot and Basic 
programs showed more frequent utilization of printed than of audiovisual resources 
^t both elementary and secondary school levels. Also of interest , was a slightly 
greater emphasis on the use of printed materials in remedial reading than in 
regular reading programs. 

Teachers in regular and remedial math programs indicated extensive use of all 
four categories of math instructional resources. In regular programs, basic 
texts and drill-and-practice materials wer.e' used more frequently than other 
materials in Pilot' and Basic samples. Use of basic texts was particularly 
emphasized in sample Basic secondary programs. In remedial programs, on the 
other hand, use of materials other than basic texts and drill-and-practice 
materials was emphasized in Pilot and Basic progj?ams. 



"4". SUMMARY OF RESULTS:- FACILITIES, STAFF, AND MATERIALS RESOURCES 

Key features in the availability of school and classroom resources were as 
follows: 

At the elementary-school level. Pilot and Basic- schools were similar 
in overall availability of central and specialized physical facilities. 
In secondary schools, which typically have larger school plant sites 
and augmented curriculxim demands, central and specialized facilities 
wQa?e more available. *' 

• The number of students per teacher was slightly larger in Pilot 
school pairs (30:1) than in Basic elementary pairs (27:1)"; in Basic 
secondary schools, this ratio was slightly smaller (25:1). 

• Education level was somewhat Ijigher among Basic elementary school 
teachers than among Pilot teachers; it was highest among sample Basic 
secondary school teachers, as might be expected with increa^d pro- 

^ ^ *fes,sional specialization* 

• At the^ eleinentary-school level, inservice training was most heavily 
focused on* reading, with about 50% of the teachers participating. 
Training for instruction of the disadvantaged had the lowest rate of 
teacher participation (about 20%) . Basxc secondary schools reported 

^ . less total in-servicG training, but relatively more emphasis on inter- 
group relations training than elementary schools. 

• At the elementary school level, remedial reading teachers vere repre- 
sented to a greater degree than other specialized support personnel 
among total staff; at the secondary school level, remedial reading and 
math teachers were represented about equally. 



# Tl>e presence of aides for. reading and math programs was reported more 
'* frequently in Pilot schools th^an in Basic elementary schools, 

# In regular and remedial reading programs at the elementary and secondary 
levels/ printed resources were used more frequently than audiovisual 

. materials* In regular math programs at the elementary and secondary 
levels/ the use of basic texts and drill-^d-practice materials was 
emphasized. In remedial matii programs/ zhere was fairly extensive use 
of macerials other than basic texts / including drill-and-practice ' 
materials, 

# Treatment-control differences (favoring tl*ie treatment schools) were found 
in only two areas: amount of in-service training and availability of 
teacher aides. , 

B. READING AND MATH INSTRUCUONAL PJROGRAM CHARACTERISTICS 

This section of the chapter describes specific characteristics of the reading and 
math programs In the Pilot and Basic sample schools, as determined from teacher 
reports* To ensure comparability of data for the different samples / only. Pilot 
and Basic schools having programs at all three grade leyels of interest are 
included in the data summarized here. 

The descriptions that follow cover all jtreading and matJi activities/ whether 
"regular" or "remedial/compehsatory One reason for the most inclusive de- 
scription is that some schools may not make cl^arcut operational distinctions 
between compensatoxry and non-compensatory- activities/ jind may use the terms 
inconsistently • Another reason is that it is presumably the total experience 
in reading or ma^h instruction that determines the student's opportunity to gain 
in achievement, not simply those component experiences labelled as remedial or 
compensatory . 



1^ EXTENT OF STUDENT EXPOSURE 

One of Che key indicators of how< a school program actually operates (as dis- . 
tinguished from goals and^guidelines/ v;hich may indicate how the program should 
operate) is* the allocati6n of student time during the school day. A school 
administrator's statement that his program emphasizes improvement of reading 
skills may mean little unless students actually spend a sizable portion of 
their -^time in reading instruction, ^^^or this reason, an extensive effort was 
made in the evaluation to obtain data "^oo the amounts of time spent in reading 
and math instruction. , Resull^s of this ef £ort are reported in the following 
paragraphs. 

a* Student Attendance and Exposure ijog bata 

The S>tiident Attendance and Exposure Log (SAI|) .provided a record of -the jiumber of 
hours each sample student ^received instruction in xeading, math/ and intercultural 
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activities,* The SAL data were collected on each student for one week of each 
month over a period of up to seven months, providing repeated estimates of the 
student's exposure to program activities •** 

The SAL exposure categories for each individual program activity were coded to 
approximate an interval scale. The exposyre categories wei;;e^: no exposure = "0"; 
1 hour = "1"; 2-3 hours = "2,5"; 4-5 hours = "4,5"; and more than 5, hours "6." 
Summary indices for both reading and math exposure were computed by totalling 
the number of hours of instruction in each component activity of the reading 
section and the math section, and then averaging these total figures over the 
number of moaths of data obtained for each s,tudent. 

To minimize the possible distortion introduced by respondents' under- or over- 
estimation of the total amount of student exposure, intensity of exposure in 
specific reading and math activities was defined as the proportion of time spent 
in each component reading or math activity, relative to the total reading and 
math estimates. Each proportion was 'averaged over the number of months of data 
reported for each student. The proportional exposure estimates were then aggre- 
gated to the school level for the Pilot, Basic elementary ,and Basic secondary 
schools, 

•* 

Elements of reading, exposure (ccxnparable elements were defined for mathematics) 
consisted of three types of activities: (a) modes of instruction such as tutoring, 
group instruction, "pullout" compensatory reading instruction, and machine- 
mediated instruction; (b) individualized activities such as diagnostic test- taking, 
independent seat exercises, report writing, student presentations, and indiyidual- ^ 
ized reading; (c) group activities such as games and contests, 

(1) SAL Exposure Results for Regular and Remedial Reading * 

Appendix tables A-10 to A-12 summarize the SAL exposure results for all reading 
activities by evaluation group. At the elementary-school -level, students in 
paired treatment, paired contirol, and unpaired treatment schools in the Pilot 
san^ple received an overall average of 11.4 hours of total weekly reading exposure, 
and student in paired treatment and control schools in the Basic elementary I 
sample recc ed an average of 10,3 hours. Basic secondary sample students received 
an aver'age oc 8,1 hours of totaL weekly reading exposur^. 

Within the individual reading activities, elementary students in the Pilot and 
Basic samples spent *the highest average weekly percentage of time in group 
instruction (about 25%) , followed by independent seat exercises (about 20%) and 
individual reading (aibout 3,4%), Basic secondary sample* students spent the 
highest average weekly percentage of time in group instruction (about 17%) , 
followed by individual reading (15%), independent se^t exercises (14%), report/ 
story writing (13%) , and diagnostic testing (10%) , 

. ; ; . '""^N 

*Intercultural and counseling activity exposure estimates are discussed in * 
VI-C — Intercultural Program Exposure, • . 

/' \ 

**For elementary- schpol students, SAL forms were completed by tne teacher. At 
the secondary-school level, SAL forms were completed by the individual student 
^ unless he wAs absent, in which case the form was completed by the teacher. 
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Remedial reading was designated by the "pullout" compensatory heading categor^f 
on the SAL; in such activities, students were pulled out of regular classes for 
remedial instruction.* Students in both Pilot and Basic elementary schools 
spent an average of 6% of total reading time in "pullout" compensatory reading. 
In Basic secondary schools, an average of 4% of total reading ^time was spent in 
"pullout" compensatory reading. 

There were' no significant treatment-control differences in total reading activity 
exposure'. In the Pilot sample, :here was a significant difference between the 
percentage of time spent in tuto::ing by an older person in paired treatment 
schools (8..O%0 and in control schools (5.3%). While the magnitude of the dif- 
ference was quite small, this could be an indication of greater availability of 
adult, tutors, or greater utilizatibn of ,,adult resources, in treatment schools* 
In Basic elementary sample pairs,' students in paired control, schools spent a 
significantly greater percentage 'of time in independent seat exercises (21,3%) 
than did treatment-school students (19.5%). 

X2) SAL Exposure Results for Regular and Remedial Mdth* ' ^ 

The SAL exposure results for mathematics are summarized in Appendix tables 

A'-13 to A-15* Average total weekly exposure was less intense in math than in 

reading activities. At the elementary-school level, students in Pilot schools 

received an average of 8.1 hours^ of math exposure weekly, and students in Basic 

schools received 8.0 hours of math exposure. Students in Basic secondary 

schools received an average of 5,7 hours of math exposure per week* 

* 

ftt the elementary-school level, in both Basic and Pilot programs, math activities 
were focusfed on group instruction and individual seat exercises, with students 
spending a quarter of their time in the average week in at least one of these 
activities. 

-\ 

At the high-school level, math activities were also concentrated in group 
instruction (25%) and independent seat exercises (21%) * Allocation of time to • 
diagnostic test^-taking was higher at the secondary-school level (15^o) than at 
the elementary^-school level (8%) * 

Remedial math was designated by the "pullout" compensatory math activity on the 
SAL, and i;efers to situations where students were pulled out of regular math | 
•clashes for remedial instruction, ^Students in Pilot schools averaged 5%, Basic 
elementary treatment students averaged 4.4%, and Basic elementary control 
students spent an average of 1.9% of total math time in "pullout" ccxnpensatory 
math. In Basic secondary schools ^ an average of 5% of total math time was 
«spent .in "pullout" compensatory math. 



^Exposure to compensatory, activities in the course of regular instruction was 
not gauged /because of the schools* difficulty in distinguishing "ccxnpensatory" 
and "non-compensatory" activities. f 
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There were no significant treatment-control school differences in total time of 
exposure to math activities for students in the Pilot and Basic evaluation groups. 
However, Pilot -treatment Students had more exposure^ than control students to 
tutoring by an older person (9.8% to 4.8%) anid Pilot control students spent more 
time than treatment students in student presentations (5*9% to 3.4%). Basic 
treatinent students had more exposure than control students to math "pullout" 
activity (4.4%^to 1.9%), and Basic control students had more exposure to 
independent seat exercises (30.4% to 27.4%).- / 

■• - - _ 1/ 

b. Exposure Data From Teacher Quest ionne^ire^s 

The teacher questionnaire provided information at the classroom level regarding 
student exposure in activities related to the development of specific skills in 
reading and math. Categories of reading ?.ctivities^ distinguished in the teacher 
questionnaire included the following: mashing letters or words;* learning 
letter names;* developing whole-word recognitions; phonic and/or structural 
analysis; vocabulary or learning word meanings; activities related to psycho- 
physical functioning, such as activities "tp improve reading-related conceptual 
abilities, to increase attention span, and | to develop visual discrimination;* 
and auxiliary'^activities, including independent reading and writing. 

Math activities distinguished in the teach'er questionnaire included activities 
related to basic math concepts (math vocabulcury, number concepts, symbol^ and 
rules, and verbal problem solving), and basic skills (basic computational 
operations' and drills) ^ 

Ik 

(1) Exposure Results for Regular and Rerr^iBdial Reading 

At the elemeritary-school level, teachers indicated that regular and remedial 
reading programs provided relatively intense exposure to word -meanings, phonic 
and/or structural analysis, and whole-word recognitions. Relatively little 
exposure was given to auxiliary or psychos-physical activities or to word- 
attack activities (which presxamably were given emphasis at elementsary levels 
prior to the third grade). At the highrschool level, the activity* of greatest 
exposure was learning word meanings or vocabulary. No significant treatment- 
control differences were evident in the^e activities at either the elementary 
or secondary level. 
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(2) Exposure Results for Regular and Remedial Math 

In Pilot and Basic elementaory schools, students in both regular and remedial 
math classes received intense exposure to basic skills and drill activities, and 
less intense exposure to activities related to basic math concepts. In regular 
secondary^-level mSth cle^sses, intensity of exposure was high in both basic skills 
and math concepts. In high-school remedial programs, by contrast, basic skills 



*Inapplicable at the secondary school level. 
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, activities v;ere given much stronger emphasis than conceptual activities; 
evidently the remedial classes attempted, to develop or strengthen basic skills 
tha^t the students had failed to master in elementary school. There was a 
sigi^ific^ntly greater emphasis on basic skills in Basic secondary treatment 
schools than in the paired cop.trol schools* ^ 
. \ ^ 

"2. :)^STRUCTIONAL PROGRAM OBO'ECTIVES ' 

Teachers' reports of emphasis on specific instructional program objectives 
were used to define program fs>cus. In reading programs, five major obje'ptive^* 
groupings (skill areas) were identified: a') comprehension skills (develppment 
of skills in using cont.ext clues, development of comprehension skills, and 
improveir.ent of comprehension rate); b) word techniques (recognition of basic 
sight word6 and phonic and structural analysis of words); c) sensory-perqeptual- 
m otor , ^ lls (auditory discrimination, visual discrimination, and motor skills):* 
(3[) global skills (library and study skills and verbal communication; and e) 
^racty&al skills (punctuation/paragraph skills, syllabification, and oral reading) 

Math program objectives of concern related to math concepts, terminology', 
operations, • attention span, andt motivation to learn. 



a* Program Focus Resu.lts for Regular and Remedial Reading 

At the elementary school level, all five major objectives groupings were given* 
considerable emphasis. In regular reading* programs, in both Pilot and Basic 
elementary schools, the greatest emphasis was* on comprehension skills and word 
techniques. In the Pilot sample, treatment schools placed significantly greater 
emphasis on word techniques than their matched controls. At the high-school 
level, programs focused on objectives related to comprehension and global skills; 
this may indicate that the students were expected to have^ acquired at least some 
skills in word techniques before they reached high school. 

In elementary-level remedial reading programs, sensory-perceptual-motor skills, 
ccmprehension skills, and word techniques were employed. Slightly loss emphasis 
was given to global skills and practice exercises. At the high-schbol level, 
remedial reading focused on comprehension skills, but also put heavier emphasis 
on word techniques thaii regular reading programs. 



b. Program Focus Results for Regular and Remedial Math * 

r 

A consistent pattern across Pilot and basic programs, with little variability, 
indicated major emphasis on objectives related to math concepts, operations,^ 
and student motivation. This pattern was found at both the elementary and 
secondary levels. Terminology and attention span were given somewhat less 
emphasise 



*Inapplicable at the secondary school level. 
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3. INSTRUCTIONAL APPROACHES ^ . ' 

/ 

a. Regular and jfeemedial P[eading 

Regular and roPdedial reading 1:eachers were asked to indicate the major approach 
used in teaching reading classes from the following alternatives, relevant only 
*to programs at the elementary-school leVe^l: linguistic-phonetiC/ language 
experience, modified alphabet, and sight method. The linguistic-phonetic 
approach was indicated by at least tw6-thirds of both regular and remedial 
reading teachers in Pil6t and Basic programs. The next most frequent response 
was the language experience approach. Fewe:p than 10% of regular and remedial 
reading teachers indicated u6e of either the modified alphabet or sight method.' 



b. Regular and Remedial Math ^ 

Regular and remedial math teachers at both elementary and secondary school levels 
were asked, to indicate whether tlieir major approach In teaching math was "basic 
text," "programmed text and program materials," or "laboratory approach." In 
regular math programs, the most common response was the 'basic text approach, 
utilized by 59% of Pilot and 60% of Basic elementary math teachers. At the 
secondary level, about 80% of Basic regular math teachers responded with the 
basic textbook approach; the remaining responses v?ere about equally dividedj 
between programmed and laboratory appfroaches. ^ 

In remedial math programs, the most common response at the elementary school level 
(Pilot and Basic samples) was the laboratory approach. At the secondary level, 
basic texts and the laboratory approach were used equally ofter\. " 

In the Basic elementary program, significantly more treatment (65%) than control 
teachers (35%) reported use of the Iciboratory approach, and significantly more 
control (41%) than treatment teachers (13%) reported use of the basic textbook 
approach. This may suggest a more innovative or experimental approach on the 
part of treatment schools. 

tf. SIZE OF CLASSROOM G^IOUPING^ 

Classes in regular reading and math instruction at the element^y leVel generally 
contained from 21 to 25 students, while classes in Basic secondary programs 
tended to be slightly Icorger (26 to 30 students). Classes in remedial instruc- 
tion in reading and math were smaller (about 16 to 20 student;s) , presumably 
reflecting more individualized instruction. There were significant treatment- 
control differences in Basic -e^mentary remedial programs: in reading programs, 
the average class size in treatment schools (about 11 students) was' significantly 
smaller than in control schools (abaut 18 students); in math programs, treatment 
school class size (about 13 students) was significantly smaller than* control 
school class size (about 20 students). 
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|h remedial reading and math instruction there was considerable etophasis on 
indi^vidualized instrqction, while regular instructional programs at the 
elementary level emphasized ^adults and students in instructional groupings 
of between two and 10, a one-to-one relation, or pupils workiJng independently, 
Sfecondary-level programs, by contrast, were often characterized, by larger 
instructional groupings of more than 20. 



5, DEGREE OF LESSON STRUCTURE 

Degree of structure in^ the c}.assroom is of interest to Educators as a class- 
room context variable, particularly .as it may interact with different student 
characteristics to mediate student learning. Two major indicators of 
structure were examined: the extent to which instruction was organized around 
clearcut goals and lesson plaas and the ex'tent of use of diagnostic testing, 
to guide the instruction. 



a • . Organization of Instruction 

Reading and math teachers w^e asked to characterize the degree of stmotuife 
in their classes in terms of 1) use of detailed lesson plans, and 2Wthe extent 
to which* instruction was based o^ clearly defined learxiing objectiviGs. In 
regular and remedial instructional programs, the overv/helming majority of 
teachers across Pilot and *Basic programs indicated the use of both lesson 
plans and improvisation. Most, teachers also reported ^xten^ive use of clearly 
de'fine^d objectives. At the elem6ntary-school level. Pilot and Basic programs 
had'^about the same overall degree of classroom structure. Secondary-level 
reading teachers reported slightly less structure than elementary-level teachers 

b. Diagnostic Techniques 

^The uti liza tion of diagnostic techniques can be an indication of the degree.' of 
structure characterizing instructional programs in reading and math, since 
greater use of diagnostic technique^ presumably suggests a higher degree o£ 
classroom structure. Bo^th frequency of diagnostic testing in sample ochooli. 
and extent of additionral instruction i^rovided to sample students on the basis 
of diagnostic testing were examined. 

In general, both regular and remedial instructional programs in reading and 
math were characterized by nfoderately frequent admini<3trations of diagnostic 
tests, ranging from "less than once a month'* to "twice or more a week." In. 
Pilot regular reading programs, diagnostic tests were administered more often 
in treatment school classes *than in control schopl classes! '^y contrast, in 
3asic elementary remedial programs in reading and math, testing was more 
frequent in control school classes than in treatment school classes. 

The. use of diagnostic testing as a guide to further instruction was mure common 
in remedial instruction than in retjular instruction, and more frequent iu math 
than in reading. There was little difference in this regard between Pilot 
and Basic instructional programs at. the elementary level .A High-schoo} • 
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programs tended to rely less than elementary-level programs on diagnostic 
testing to guide additional instruction. In Basic elementary remedial math 
programs/ additional instruction was based on diagnostic testing to a 
significantly greater extent in treatment classes than in control classes. 



6. SUMMARY OF RESULTS: READING AND MATH INSTRUCTIONAL PROGRAM CHARACTERISTICS 

The instmactional program characteristics of Pilot and Basic sample groups are 
important in the evaluation as process variables that may influence learning 
and as variables that may reflect the utilization of ESAA funding resources, 
>Some of the more consistent findings in the data on these instructional programs 
axe featured below. 

• Sample Pilot students received slightly greater average total weekly 
exposure in reading instruction (11.4 hours) than sample* Basic elementary 
students (10.3 hours) ; exposure to math instructional activities was 
similar for sample Pilot {Q.l hours) and sample Basic elementary 
students (8.0 hours) • At phe secolidary level, total weekly exposure to 
reading activities was 8.1*^Hours and exposure to math activities was 

5.7 hours. ' • 

• Particular reading activities that elementary teachers emphasized in 
the classroom included vocabulary / and phonic and structural analysis; 
at the secondary level, teachers emphasized vocabulary. Activities in 
elementary and secondary math programs were focused on basic math 
concepts and on basic skills and drill activities. 

• Objectives emphasized the most in-^regular reading programs were related 
to improving comprehension skills and basic word techniques; remedial' 
reading programs focused on development of sensory-perceptual motor 
skills, comprehension skills, and word techniques. Regular and remedial 
math program objectives were focused on math concepts and operations and 
on motivation to learn math. 

• The linguistic-phonetic approach was the major teaching approach used in 
regular and remedial reading programs at the elementary level. Regular 
math programs were characterized by the basic text approach. Use of 
the laboratory approach was more common in remedial math programs at 

the elementary level, while the major teaching approaches at the seqpndary 
level were programmed learning and laboratory. 

• The sizes of regular elementary-level classes in reading and math 
generally ranged from 21 to 25 students; at the secondary level, 
classes in reading and math were slightly larger. Remedial reading 
and math gave greater emphasis than the regular instructional programs 
to individualized instruction. ^ 

• Both regular and remedial reading and math teachers reported the use of 
lesson plans and improvisation and the use of clearly defined learning 
objectives. Diagnostic tests were administered more frequently in math 
progreims than in reading programs; reliance on diagnostic testing to guid'fe 

. additional instruction was more characteristic of ^remedial .than regular 
instructional programs. 

B7 
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There were statistically significant treatment-control group differences in a 
few instructional program features. For example, in Pilot sample reading and 
math/programs, a significantly higher proportion of time was spent in tutoring 
by ..n older person in treatment schools than in control schools. Also, 
in Basic elementary remedial reading and math programs, instructional class 
size was significantly smaller in treatment schools than in control schools. 
However, the number of significant treatment-control differences was small 
considering the many different combinations of dimensions on which the treatment 
and control groups were compared. A certain number o¥ apparent differences 
could easily have been found on the basis of chance alone, given that treatment- 
control comparisons were made on a large number of program variables ^or three 
different sample groups (Pilot, Basic elementary, and Basic secondary) , two 
different subject areas, several diff erent , grades, and regular and remedial 
instruction. Thus, despite occasionsil differences in the data for treatment 
and control schools,, there is only limited evidence of any widespread, or _ 
systematic pattern of differences^ This finding suggests that it may be unreal-- 
istic to expect large or systematic differences in the achievement outcomes for 
the two experimental groups during the initial school year of the evaluation. 



C. INTERCULTURAL PROGRAM EXPOSURE 

Intercultural programs in sample schools were defined as activities involving 
inter-racial interaction and cultural enrichment activities. One of the impor- 
tant objectives of intercultural activities as a component of* ESAA Basic pro- 
grams was to facilitate desegregation in Basic schools. In addition, some Pilot 
schools apparently used intercultural. activities to provide -cultural enrichment 
in minority-impacted schools. Because of difficulties in definilnig* sufficiently 
sensitive program descriptors relevant to these objectives, discussion of inter- 
cultural activity in this report is largely limited to a consideration of the 
percentage of students' time spent in intercultural programs. Intercultural 
program characteristics will be given increased consideration and emphasis in 
.the second-year evaluation, drawing upon in-depth site studies and the school 
climate questionnaire. 

Average weekly exposure estimates in non-academic activities were obtained 
from Student Attendance and Exposure logs in each of the following acti- 
vities: intercultural activities such as interracial programs, cultural 
enrichment programs, and ESL (English as a second language) or bilingual 
activities; field trips; and "other" activities including individual and 
group counseling/guidance. Relative emphasis on different intercultural 
and "other" activities was defined in terms of the average proportion of 
time spent in intercultural/other activities over the number of months 
reported for each student. These data were then aggregated to the school 
level for Pilot and Basic sample schools. 

Appendix Tables A-16 to A-18 present the results for the intercultural exposure 
data. At the elemontary- school level. Pilot sample students received an average 
of 2.4 hours of total weekly exposure in intercultural/other activities, while 
Basic sample studervts |iad an average of 2.0 hours' exposure. The Basic secondary 
school students received an average of 2.4 hours* exposure in intercultural/other 
activities. Among the, intercultural/other activities, both Pilot and Basic saiQpl 
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schjols placed heaviest emphasis on cultural enrichment programs (33% and 41%, 
resp3§l4vely) • Field trips (22% of total intercultural/other exposure in Pilot - 
schools, 16%' in Basic schools) and group pounseling (18% and 20%, respectively) 
were next in relative emphasis. At the secondary level, the major concentration 
was on field trips and counseling/ and less ^emphasis was given to intercultural 
and interracial programs • 

D. WAS THE PROVISION OF SERVICES MATCHED TO NEEDS? 

Evidence was presented in Chapter IV that students in the* sample schools had 
severe academic needs , as indicated by tfieir depressed pretest scores in reading 
and math achievement tests. Furthermore, there were at Ijeast tentative indications 
that districts' program funding was directed toward students having academic needs, 
as well as toward schools' with socioeconomic needs." At this point, it appears 
useful to assess whether ESAA dollars, specff ically^ were translated into services 
that were targeted *to meet identified academic needs • ' To explore this question, 
only ESAA-fiinded (treatment) schools were examined. 



1. OPERATIONAL DEFINITIONS 

Academic need was defined by school achievement pretest subscale scores in r^eading 
vocabulary and comprehension and in math concepts and* computation.* 

Intercultural need was not defined at the time of pretest. Therefore, phe question 
of whether intercultural programs met student needs vas not addressed in the first 
year's evaluation. 

Extent of services provided was defined in terms of various instructional program 
indicators gauging each student's experiences in the classroom. Estimates of 
average total weekly exposure in reading and math programs were obtained from 
the Student Attendance and Exposure Logs. Teachers provided data on programs 
at the classroom level* Suinmary indicators were then generated to measure 
emphasis on * comprehens ion , word techniques, and vocabulary skills f as well 
as program emphasis on basic math concepts and basic operational skills. 

For analysis purposes, both student pretost scores and instructional program 
focus indices were then aggregated to the school level for each grade. Only 
treatment schools that were in treatment-control pairs and that had complete 
data were considered in the analysis. 



*Math problem^ solving was an additional subscale at the socondary-school level. 
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2. METHOD \ 

Pearson correlation coefficients were computed between each program indicator 
- . and each pretest subscale score among treatment schools. Appropriateness of 
services provided was assumed to be indicated by the magnitude of negative 
correlation between pretest subscale scores and corresponding program intensity 
indices. A highly negative correlation would indicate that the receipt of 
services in a particular program activity (e.g./ vocabulary skills training) 
was given to students who showed the greatest achievement deficiency in the^ • 
corresponding skill area (S.g., vocabulary skills). 



3. RESULTS 

Tables VI-2 to present the results of analyses of the matching of 

services to students* needs. Caution must be observed in interpreting 
these results, because of the superfiqial nature of the analyses reported 
here. Given this limitation, however, it appears that, in the sample 
elementary schools, services were well targeted to meet identified stu- 
dent needs. In Basic elementary programs there was a fairly consistent 
pattern, across grade levels, of significant negative correlations be- 
tween average total weekly student reading exposure and school-level 
reading pretest achievement scores; similar -negative correlations were 
found between math exposure and math pretest subscale scores, 

TABLE VI-2, CORRELATIONS BETWEEN ACADEMIC NEEDS AND SERVICES 
RECEIVED IN BASIC ELEMENTARY PROGRAMS 







Pretest Subscales 


Level 


Type o^ 
Service 


Reading 
Vocabulary - 


Reading 
Comp reh ens ion 


Math 
Concepts 


"Math 
Comprehension 


Grade 3 
Total N = 


V Reading 
' Math 
32 


-.34* 
-,40* 


-.28 
-.37* 


-.40* 
-.38* 


-.22 
-.05 


Grade 4 
Total N = 


Reading 
Math 

31 


-,45* 
-,50* 


-,39* 
-,41* 


-,37* 
-.37* 


-.29 
-,36* 


Grade 5 
Total N = 


Reading 
Math 

31 


-,39* 
-.25 


-,39* 
-,25 


-.40* 
-^27 


-,37* 
-.31* 



*Significant at the a == ,05 or above* 
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TABLE VI-3/ CORRELATIONS BETWEEN ACADEMIC NEEDS 
AND SERVICES RECEIVED IN PILOT PROGRAMS 



Type of 
Level Service 


Pretest Subscales 


Reading 
Vocabulary 


Reading " 
Comprehension 


Math 
Concepts 


Math 
Comprehension 


Grade 3 Readinq 
Math 

Total N = 14 


-.39 
" -.48* 


-.26 
-.17 


-.25 . 
-.26 


.09 
-.19 


drade 4 Reading 
Math 

Total N = 17 


-.20 ' 


-.19 
-.36 ^ 


.14 
-.27 

•1 


0 

« -.24 


Grade 5 ^ Reading 
Math 

Total N = 16 


* 

-.15 
-.16 


-.24 
-.14 


-;19 
-.12 


-.36 
-.41 



♦significant at the a = .05 level or beyond. 



TABLE IV-4. CORRELATIONS BETWEEN ACADEMIC NEEDS AND SERVICES 
RECEIVED IN> BASIC SECONDARY PROGRAMS 







Pretest 


Subscales 




Type of 
Level Service 


Reading 
Vocabulary 


Reading 
Comprehension 


Math 
Concepts 


Math 
Comprehension 


Math Problem 
Solving 


Grade IQ. 












Vocabulary Skills 
Math 


-.63* 
.1 


-.61* 
.02 


-.55* 
.02 


-.58* 
.06 


-.04 
-.63* 


Total N = 11 










o 


Grade 11 












Vbcabulary jskills 
Math 


.-.26 
-.25 


- -.37 
-.33 


-.46 . 
-.44 


-.40 
-.33 


-.14 
-.28 


Total N =i- 10 












Grade 12 












Vocabulary Skills 
Math 


-.58* 
-.40 


-.70* 
-.45 


-.40 
-.32 


-.55* 

-.28'' 


-.24 . 
-.03 


Total N = 11 










4 



♦Significant at the a = .05 level or beyond. 
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since the various pretest scores were strongly inter correlated , the data suggest 
that students with the greatest needs in g^Srieral received greater reading and 
math Instructional exposure in ESAA Basic progams» Similarly r in Pilot programs ^ 
although the number of cases was smalj^ the pattern across grade levels shows 
average total weekly exposure in readitig and math to be negatively correlated 
with school level pretest achievement scores. 

In summary, previous chapters indicated that sample studehjts ill ESAA-funded^ 
districts and schools had acute educational needs • The present chapter presented 
tentative- evidence that ESAA dollars were translated into services having at 
least a general correspondence to specific academic deficiencies as measured 
by the students' achievement test scores. 
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CliAPTER VII 
BASIC PROGRAM DESEGREGATION 



One of the primary activities presumably characterizing districts receiving 
ESAA Basic grant funds was desegregation of the school system (although in 
many districts this process was started well before ESAA funding) . •This 
chapter focuses on desegregation activities in the sample Basic programs; 
it does not consider desegregation activities in the Pilot schools # which/ 
by definition, were minority-isolated. 

The following descriptions center on the district level of operation and 
policy formation for desegregation plans and activities. Two primary sources 
of desegregation information were used: .survey data from the Office of pivil 
Rights (OCR) and questionnaire responses 'from the district superintendents in 
the 1973-1974 ESAA evaluation sample. The Office of Civil Rights is respon- 
sible for administering Title VI of the 1964 Civil Rights Act. In compliance 
with this law, OCR conducts an annual survey of school systems to ascertain 
whether the school systems have achieved proper racial balance to the 
extent possible. 

A merged data set containing' both types of desegregation information (ESAA 
evaluation .data and OCR data) was constructed for 46 Basic districts in the 
ESAA evaluation sample. This is a .reasonably large sample/ and fair con- 
fidence can probably be placed in data summarized across all 46 cases. 
However/ where results are broken down by HEW region or cross-classified 
along two or more dimensions/ the number of districts in any one cell often 
becomes quite small/ and the results should not be interpreted too literally 
as representative of all other districts in that same cell- 

Two additional constraints on the generalizability and meaningfulness of the 
data should be mentioned at this point. First/ the types of desegregation- 
related data reported here are fairly superficial. The data tell us something 
about district-level policies and characterize resulting activities in overall 
statistical termS/ but they do not provide a detailed picture of how staff and 
sjiudents reacted to and participated in the desegregation activities. Better 
and. more detailed desegregation-related data should be available when the 
second-year reporit is prepared, because of the pretest and posttest admini- 
stration of the newly developed School Climate Questionnaire. Furthermore./ 
the in-depth study of 30 districts performed in the second evaluation year 
should help to provide greater insight, of a case-history nature, into 
desegregation-related policies/ activities/ and attitudes in the 30 sites. 

A second, constraint relates to the sample on which the desegregation data are 
based. Because of problems encountered in obtaining release of test and 
questionnaire data/ the first-year sample did not include results from i:wo of 
the largest schopl districts in the original evaluation -sample/ both located 
in the eastern portion.^of the country. One of these districts had a very high 
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percentage of Black students and the other had the largest number of Spanish- 
descent students of any district in^the study. These exclusions reduce the 
generalizability of the findings somewhat ^ although the sample still includes 
some fairly large districts from other parts of the country. It is hoped 
that data from the two districts concerned can be included in the results for 
the third-year evaluation. 

A. DEFINITION OF TERMS 

Before summarizing the desegregation data, it may he useful to define some of 
the terms used by the Office of Civil Rights (OCR) to define the status of 
districts and schools with respect to the nature and progress of their 
desegregation activities. The following discussion considers the interpre- 
tation of various OCR "compliance codes" and the derivation and meaning of 
the OCR desegregation index (D.I.). 

In monitoring the desegregation activities in school districts , OCR assigns 
one of four types of compliance codes to each district. These codes are 
intended to indicate the present status of the desegregation efforts in each 
district. 

Voluntary . The school system has not completely abolished its dual school- 
system structure, but the district office has developed its owrt plan for 
complete desegregation. Such desegregation plans have generally been drawn 
up in order to avoid termination of Federal funds by HEW under Title VI 
of the 1964 Civil Rights Act. 

Court Order . The school system has accepted the terms of a final court order 
for desegregation and has submitted an assurance to HEW that it will abide 
by the regulations specified within the final court order. 

Litigation . The school system does not have a final court order for desegre- 
gation and is in the process of negotiating settlement. A final court order 
has not been submitted to HEW. 

441. The school system has achieved satisfactory desegregation and has 
eliminated any dual school system within the district according to 
regulations specified by OCR. 

The government-accepted OCR district desegregation index attempts to measure 
the extent of desegregation in the schools within a dist;rict; the index ranges 
from O.O (a totally segregate(J environment) to 1.0 (a uniform racial distri- 
bution) . It compares the racial balance of each school in the district with 
the overall racial composition of the district. Racial balanpe, insofar as 
it can be achieved with a particular district's student population ^ is viewed 
as the best situation. The farther the racial balance of each school is from 
the overall racial distribution of the district, the lower the index value. 
It should be enphasized again ^ however ^ that the .D.I. is a rather gross 
measure that does not reflect any of the dynamic interactions within a district, 
nor does it reveal underlying iitvotivations or attitudes concerning desegregation 
and integration. 
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The formula for calculating the desegregation index is as follows; 



n 



a. X b. 

3 '3 



\ 



D.I. = 




X B 



t 




where : 



a 



. = nmnber of minority students in school j 



b. = number of non-minority students in school j 
D 

A^-= total number of minority students in district 

B*^ = total n\amber of non-minority students in district 

j = school indicator 

n = number of schools in the district 



Fifty percent of the Basic districts had been desegregating from one to five 
years. An additional 35% had been desegregating from 6 to 10 years, while 
13% of the districts had been desegregating frpm 11 to 20 years. Only two 
districts in the evaluation sample began desegregation activities in 1973. 



Of the 23 districts that had been desegregating from one to five years 
and the two that began in 1973 , 65% were operating under the court- 
order compliance code. There were equal numbers of districts (six) operating 
under voluntary, court-order, and 441 compliance codes in the 6- to 10-year 
desegregating category,. The most frequent type of compliance code for 
districts that had been desegregating from 11 to 20 years was the 441 (43%) . 
From these figures on the period and type of desegregation, it appears that 
rather than encouraging new voluntary desegregation, ESAA money was used 
primarily to meet needs associated with on-going desegregation, at least in 
the first year of ESAA funding. 

Table VII-1 illustrates the relationship between the number of years a district 
had been desegregating and the percentage of minority students in the district 
in Fall 1973. The total nimiber of students involved in each cell is also 
specified (except for one district in which enrollment data were not yet 
available). Almost all of the districts with moderately high percentages of 
minority students had been desegregating from one to five years. Among 
districts -with low percentages of minority students, by contrast, over half 
had been desegregating for more than five years. 

The most frequent types of compliance codes for the 46 Basic evaluation 
districts were court order (48%), 441 (30%), and voluntary (22%). Only one 
dis|:rict was operating under a litigation code. The percentage of minority 
students in the district was not related in any consistent manner to the type 
of compliance code assigned to the district. 




\ 
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TABLE VII-1. NUMBER OF YEARS DISTRICT HAD BEEN DESEGREGATING 
IN RELATION TO PERCENTAGE OF MINORITY STUDENTS 
IN DISTRICTS IN FALL 1973 



Years of 


Low Percentage Minority (l0%-39%) 


Desegregation 


Number of 


Total Number of 


Activity 


Districts 


Students Involved 


0 


0 


0 


1-5 


10 


335,893 


6-10 


14 


237,623 


11-20 


4 


^72,959 




Years of 


Moderate Perc/intage Minority (40%-69%) 


Desegregation 


Number of y 


Total Number. of 


Activity 


Districts 


Students Involved 


0 


0 




1-5- 


11 


684,312 


6-10 


1 


28,830 


11-20 


1 


21,646 



Years of 


High Percentage 


Minority (70%-100%) 


Desegregation 


Number of 


Total -Number of 


Activity 


Districts 


Students Involved 


0 


1 


6,429 


1-5 


2 


96,748 


6-10 


1 


4,807 


11-20 


1 


17,320 


Total Number of Districts = 46 



Court-order compliance^ codes were most likely to occur in the mid-Atlantic 

and Southeastern 'states. The Metropolitan Northeast and North Midwest 
regions were most likely to be operating under 441 codes, while the voluntary 
compliance codes occurred with equal frequency in the Metropolitan Northeast, 
Southeast, and South Central regions. Caution should be observed in attempting 
to generalize from these data, however, because of the small number of sample 
districts in some regions. 



Table VII-2 shows the range of the 1973-74 district desegregation index (D.IO 
values for the 46 sample Basic districts, and the total number of students 
enrolled in the districts having the various D.I. levels. For purposes of 
this discussion, D.I. values ranging from .85 to ^1.0 are considered to 
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indicate a \igh degree of desegregation, .55 to .84 a moderate degree, and 
.25 to .54 a low degree. As shov/n in the table, -over 80% of the districts 
had modera.te- to-high desegregation, with over half the districts (61%) in 
the high category at the beginning of ESAA implementation. 



TABLE VII-2. IU\NGE OF THE 1973-1974 DISTRICT DESEGREGATION 
INDEX VALUES FOR THE SAMPLE BASIC DISTRICTS 





Number of Basic 


Percentage of 


Total Number 


1973-1974 District 


Districts in 


Basic Districts 


of Students 


jj* X • vaxuc 


c«qL tecjorv 


JLil Vi«a Uc^^v^i. y 


XII UlS Ux jlc 


.95-1.00^ 


18 


39.1 


220,173 


.85- .94-"^^^ 


10 


21.7 


192,710 


.75- .84 

\ 

.65- .74 Moderate 
.55- .64 


4 
5 
2 


8.7 
' 10.9 

4.3 ^ 


182,028 
^ 306,857 
82,504 


.45- .54^ 
.35- .44 Low 
.25- .34 ^ 


2 


4.3 


109,771 


2 
3 


4.3 
6.5 


" 283,993 
128,531 


Total Number of Districts = 46 







There' was a statistically significant relationship (.01 level) between the 
total enrollment of districts arid their 1973-74' D.I. values. « Districts with 
low enrollments had higher D.I. values, than districts with moderately high 
and high enrollments. No signif ican^:^ relationship was found between the D.I. 
values and either the number of years the districts had been desegregating 
or the HEW region in which the districts were located. 

Table VII-3 illustrates the reflationship between the D.I. values and the 
district's percentage of minority students. Racially balanced districts 
(i.e., districts with high D.I^ values) were most likely to have low per- 
centages of* minority students and small total enrollments. Moderately 
high-minority-percentage districts with large total-enrollment figures 
tended to have low or moderate D.I. values, reflecting lack of racial balance 
in the schools. Although there were very few cases in the high-minority- 
percentage category, /the overall pattern suggests that schools with moder- 
ately high or high .percentages of minority students had more problems as- 
sociated with desegregating their school system than did schools with low 
percentages of minority students. Low, moderately high, and high minority- 
percentage districts may have faced very different problems in meeting their 
desegregation needs, especially as many of the high-minority-percentage dis- 
tricts had only recently begun to desegregate their school systems. 

J 
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TABLE VII-3. RELATIONSHIP BETWEEN 1973-1974 DISTRICT D^I. VALUE 
AND DISTRICT'S PERCENTAGE OP MINORITY .STUDENTS IN PALL 1973 





Low Percentage Minority (10%-39%) 


••*1973-1974 
* * D.I. Value 


Number of 
Districts 


Total Number of 
Students in District 


.257.54 
.55-.84^ 
• 85-1.00' 


5 
21 


78,934 
255,881 ^ 
311,660 




1973-1974 ' 
D.I. Value . 


Moderate .Percentac 


re Minority (40%-69%) 


Number of 
Districts 


^ Total Number of . 
Students in District 


.25-. 54 
.55-. 84 
.85-1.00 . ' 


4 
5 
4 


355,236 
298,188 
81,364 




1973-1974 
D.I* VALUE 


Hiqh Percentage Minority (7a%-100%) 


Nvimber of 
Districts 


Total Number of 
Students in District 


/ .25-. 54 
.55-. 84 
.85-1.00 


1 
1 
3 


88,125 
17,320 - 
19,8^9, 


« 

Total Number of Districts = 46 
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A word of caution should be added at this point regarding samj)le represent^- 
tiveness. As noted earlie;r, the evaluation sample districts, ihough^ highljf 
representative of the defined evaluation universe, included a ^onfewhat larger 
proportion of large districts than the ESAA award universe. In addition, and 
perhaps more importantly for the present discussion, the sample Basic districts 
averaged^ significantly lower beginning-of-year D.I. values than districts in 
the ESAA Basic award universe; this indicates tha^ sample districts tended to 
start the evaluation year with poorer racial balance across schools than the 
award universe. Thus, results of the desegregation analysis in sample Basic 
"districts cannot be generalized with complete confidence ^to the Basic award 
universe. 

In summary, the results indicated several significant patterns. Recency of 
desegregation was related to both the enrollment* size of the. district and the 
proportion of minority students within the district*. > Districts witii large 
enrollment or a high percentage of minority student's were likely '^o have 
been desegregating for only a few years. Sample, districts ITTth relatively 
good racial balance across scRools (D.I.* values in the range .85 to l-.O) 
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typically had small total-enrollment ^figures and*a low proportion of' minority 
•students. ' ^ 

C. SPECIFIC ELEMENTS OF DESEGREGATION PIANS 

Data collected from district superihtendent^in the ESAA Basic evaluation 
sample provided descriptions oC^the district desegregation plans. Results 
reported here are based on six jtjuestionnaire items. ^-4^§7^*f^he items presented 
a list of possible administre^'^ve procedures, and asked the superintendent to 

.check all of the procedures being used in his/her district to achieve desegre- 
gation; among the alternatives listed were bussing, .establishing schools with 
limited grade levels , ^ pairing or closing schools, and drawing new school 
boundaries. Another item listed several procedures that might be used to 
assign students in elementary and secondary schools, and asked the superintendent 
to specify which one procedure best described the practices in his/her district; ^ 
alternatives included assignment by strict geographic .area, use of geographic 
assignments but with allowances for voluntary transfers, assignment on the 
basis of intelligence or achievement, free choice of assignment within district, 
and bussing to achieve greater racial balance. The remaining four items asked 
what percentages of white (Anglo) and minority elementary and secondary students 
attended, for desegregation purposes, a school not the school nearest to their 

. home. , " ' 

In interpreting the superintendents' responses, twovconstraints must be con- 
sidered. First, the major data on desegregation practices are bas^d on only 
tvo questionnaire items, and secpnd, the sample sizes were somewhat small. 
Although there wefce 46 Basic elementary districts, tla number of superintendents 
selecting any^^poxticular response was often much smalJL<ir. For example, only 13 
superintendents of Basic districts specified bussing as the primary assignment 
procedure at the elementary level. This makes it difficult to draw general 
inferences about the characteristics of districts that used bussing for desQsre- 
gation purposes. ' 

When asked to specify all elements of their districts' desegregation plan^, most 
superintejidehts (78^%) selected the alternative, "Drawing, new school boundaries 
to provide for desegregation." Another commonly designated procedure (70%) was 
"bussing students from other areas,-" as shown in Tables VIIr4 and VII-5, responses 
to other litems indicated that larger percei^tages of minority students than of 
white s,tUdent§ attended non-nearest schools, though the difference was smaller 
at the elementary level than in high school „ Approximately 59% of the 
superintendents reported the closing of certain schools so as to force students 
of different racial and ethnic groups to attend the remaining schools, jilmost 
half the superintendents reported the establishment of schools with limitjed g^ade- 
levels (e.g., grades K-2 or 3 through 6), and a third reported the pairiiig of ^ 
schools; both of 'these procedures, like the closing of schools, are designed to 
cause students from wider geographic areas and broader racial/ethnic spectra to 
attend the same schools. , . / ' 

On the item asking ^bout the one most comlon student assignment pj^^ctice, the 
most frequent response at both the elementary level (47^) and the secondary 
level (62%) v^s assignment by geographic area. Th6 next most common response 
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• (38% and 27% at elementary and secondary levels, respectively) , was the use of 
bussing. Relatively few superintendents specified other techniques, and none 
indicated the use of intelligence or achievement as the primary assignment 
criterion. 

Several interesting relationships were found between the types of des^regation 
methods employed/ and certain district characteristics. For example, districts 
that listed bussing among the elements in their desegregation plans, tended to 
have low percentages of minority students and/or large district enrollments. 
There was^also a slight tender\cy for bussing to be associated with high Desegre- 
gation Index values, hut this pattern^ was somewhat. equivocal. 

\ 

Of particular interest, because of its direct relevance to the bu^s;Lng issue, ^ 
was the item askitig about the district's primary assignment procedure. Dis'tricts 
designating bussing as their major assignment practice showed h, relatively strong 
and consistent pattern of high Desegregation Index values (i.e. , good racial 
balance).* In additid|, such districts, tended to have low percentages of 
minority students and fmall enrollment sizes. These relationships were stronger 
at the elementary level than at^ the secondary level. • 

* The drawing of new school boundaries so as .to provide better balance in the 
proportions of students of different racial/ethnic groups assigned to a given 
school was also a common practice. Districts utilizing this method generally 
had low percentages of minority students and larger district enrollments. 
Furthermore, these districts were typically serving medium-sized or la^rge 
coiranunities rather than small towns. These results are not too surprising, 
since in a small district or community with few schools, there is likely to 
be much less flexibility in the arrahgment of school boundaries. 

Although the remaining desegregation techniques were use;i less often, ther^ 
weire sctfne noteworthy relationships. Distr-icts most likely to use the closing 
of schools were those with relatively good racial balance across schools (high 
D.I. values). Of the districts indicating the use of assignment of students 
by geographic area, 30% had low D.I. values (.25-. 54) and approximately- 70% were 
operating under court" orders . The pairing of schools and the establishment of 
schools with limited grade levels as desegregation methods were not related in 
any meaningful way to any district characteristics. 



*The stronger relationship between bussing and the Desegregation Index for this 
item, as compared with the item inquiring about all' elements of the districts ' 
desegregation plans, reflects differences in the composition of districts 
designating bussing on the two items. Thirty-two districts listed bussing as 
.one element in their desegregation plans, vhile only, 13 listed it as the major 
assignment practice. These latter 13 districts were a subset of the 32 districts 
but differed from the remaining 19 districts in STstrict size and percentage of 
minority' students. Overall, the 13 districts were smaller jand had smaller per- 
centages of minority students than the other 19 districts. 
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D. AMOUNT OF DESEGREGATION IN ESAA YEAR I (1973-1974) 

The amount of desegregation occurring in the 46 sample Basic districts "during 
ESAA Year I was calculated by taking the difference between the 1973-1974 and 
1972-1973 D,I. values for each district. It should be noted that the D,I. 
values show the districts' status at the start of , the school year; thus, the 
differences in the D.I. values for the two years represent rhe differences 
between approximately October 1972 and October 1973, A more meaningful com- 
parison, reflecting changes made during the first year of ESAA implementpition, 
would be a contrast of D.I, status in October 1973 and October 1974, Such a 
comparison will be provided in a subsequent report after the second-year data* 
Tiave* been analyzed, 

« « 
Analysis of the D.I, values for October 1972 and October 1973 indicated that 
little' change in desegregation occurred in most distriot^ during that period. 
There were some slight positive changes (toward greater desegregation) in 
24% of the districts, relatively large positive changes in 510%, no change in 
28%, and some slight negative 'changes in 28% of the districts. These findings 
are somewhat expected, and tend to reinforce the contention that ESAA funds 
were used largely to meet needs incident to reduction in segregation and 
discrimination, rather than to initiate desegregation. 

Two disticicts in the 1973-1974 ESAA evaluation sample did begin delfe^egregation 
activities directly before the beginning of the 1973-1974 school year, part 
of the ESAA funds in those districts may have been* used to assist in desegre- 
gation activities, so that the classrooms and schools within the district 
would be more racially balanced before the 1973-1974 school year. One of the 
two districts had an appreciable gain in its district D,Ii. value (,4); its 
characteristics are described more fully below. The other district also ^ - 
made a small positive gain (.03) in its D.I, value. 

As noted above, 10 distrir :s made fairly sizable gains in desegregating their 
schools. These gains in D.I, values ranged from .1 to .5, with seven districts 
concentrated at the ,1 level and one district each at. the ,2, .4, and ,5 levels 
Tables VII-6 and VII-7 describe the characteristics of these districts with 
respect to HEW region, ni:iinber of years desegregating, conpliance code, percen- 
tage of minority students in the districts in Fall 1973, and the enrollment 
and total number of minority students in each of the districts for the 1972- 
1973 and 1973-1974 school years. Enrollment, figures were not available for 
the district with the .4 gain in district D.I,""* value. 

The majority of these 10 districts were in either the South Central or South- 
east regions of the country, and were operating under court-order compliance 
codes. The three districts making the largest gains in desegregation were all 
operating under court orders. Three of the 10 districts were operating under 
441 compliance codes, and they tended to make smaller positive gains. The 
number of years each district had been desegregating varied widely from zero 
to 20 years. Low, moderate, and high minority-percentage districts were all 
represented in the 10 districts making noticeable positive gains in desegre- 
gating their schools. 

' . 113.. . . . 
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£• SUMMARY 

Overall/ little new desegregation was found to have occurred in most of the 
' sample E^AA Basic districts between October 1972 and October 1973, although 

20% of the districts did show an appreciable gain in desegregation index (D.I.) 

values over that period. The probable explanation for the lack of gain in most 
" distriG^ts. is. that" a substantial , portion of the districts started with moderately 

high o4 high D.I. values in October 1972, and most had already been desegregating 

for several years before the start of the ESAA program. Taken together with 
lothex data on the districts' desegregation plans, these results sugge&t that 
"''most of the sample Basic districts used their supplemental funds to meet needs 

related to desegregation activities already implemente.d or in process, rather 

than to start new desegregation activities. 

The most common means taken by districts to desegregate were redrawing school 
boundaries and bussing students from other areas. These methods were used 
especially. by districts with large enrollments with relatively small percentages 
of minority students (e.g., 10% to 3?%). There was also soirfe evidence that 
bussing was associated with districts that had greater racial balance (higher 
D.I. values). Somewhat less often reported, but still fairly common, was the 
closing of some schools or the restriction of certain schools to a smaller 
range of grade levels-«both of these being methods of drawing from larger geo- 
graphic areas and' forcing students of different racial/ethnic groups to attend 
the same school 1 
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CHAPTER VIII 
STUDENT GAINS IN ACHIEVEMENT TEST SCORES . 



Previous chapters of this report have indicated the academic needs of students 
in Basic and Pilot schools in terms of their performance on reading and math 
tests, as well as the generally low socioeconomic level of the students' 
families. Also evidenced were the needs of the sample districts and schools 
in terms of their concentration of disadvantaged students, as well as special 
needs related to degjEtgregation or to minority-group isolation. The earlier 
chapters further indicated an apparent emphasis of the programs on meeting 
legislative objectives, and a focus of resource allocations (dollars and 
services) on school and student needs • The present chapter is concerned 
with the effects v'^f those resource allocations on the achievement of students 
in ESAA-funded districts. 



A. TREATMENT-CONTROL COMPARISONS OF ACHIEVEMENT TEST GAINS 

This section describes the results of analyses of the differential effects in 
student outcorae associated with the manipulation of the experimental variable — 
the presence or absence of ESAA funding. All of the analyses reviewed in this 
chapter have as their ^fundamental focus the possible effects of ESAA funding 
on student achievement outcomes. Fpr, the remainder of this chapter, the term 
"impact" refers specifically to analyses dealing with differences between ESAA- 
funded (treatment) and non-funded (control) schools. 

Before considering the actual methodology and results of the experimental 
analyses, it is important to review three factors greatly affecting the 
relevance of the findings for educational policy. First, the pretest-posttest 
interval was quite short. The average interval was less than 5-1/2 months, -with 
93% of the schools being posttested in a period of five tp six months after 
pretest. There was little time for the ESAA funding to exert its influence on 
student performance, especially since, in this first year of ESAA, many dis- 
tricts were late in full-scale program implementation. .Second, in many of the 
sample districts there was less than 5% difference in total funding between the 
treatment and control schools, and in a few cases, the control school actually - 
had greater funding. Finally, there were extremely few systematic or pervasive 
program differences between treatment and control schools. In other words, the* 
control schools tended not only to have nearly as much money as the treatment 
schools/ but also to spend that money for similar kinds of program activities. 

These findings greatly reduce any expectation of obtaining consistent treatment- 
control outcome differences in the first year of the evaluation. Further, they 
suggest that little weight should be given to the first-year impact analysis, 
findings in formulating educational policies related to the ESAA program. It 
seems fairly clear that for the first-year ^iesults, at least, less interpretive 
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emphasis should be given to analyses of the effects of ESAA dollars per se , 
and greater emphasis should be placed on discovering the types of program ' 
activities (regardless of the source of funding) that were more beneficial to 
ESAA-eligible students- (This latter approach is represented by the exploratory 
analyses described in Chapter IX) • 

Despite the difficulties anticipated in interpretjlng the impact analyses, it 
was felt tHat those analyses could possibly reveal some findings of interest. 
The remainder of this first section describes the impact analyses and their 
outcomes • 

■> • . - ^ 

1. EXPERIMENTAL PARADIGM 

The analyse perfonned to evaluate the impact of ESAA funding were derived from 
the experimental design of the evaluation study. In this design, a pair of 
schools, similar in ethnic composition, prior achievement level, and socioeconomic 
level, was selected from each sample district. Members of each pair were then 
randomly assigned to treatment (ESAA-funded) and control (non-funded) conditions. 

The achievement test data are characterized within the framework of a two- 
dimensional array where the treatment-control distinction accounts for one dimen- 
sion and the distr'icts themselves account for the other. The entries in each cell 
are the outcome measures against which impact is to be evaluated. For purposes 
of these analyses, the data are the values of the inc^ividual outcome measures, 
collected on each student but aggregated to the level of the. school. 

Outcome measures used for these analyses were adjusted posttest achievement 
scores. The premise behind the use of thesj^ adjustments was that each stu- 
dent's posttest performance was determined by two sets of factors, one set 
consisting of personal and background characteristics plus pre-evaluation 
educational experiences and the other set reflecting the student's educa- 
tional experiences during the evaluation period. The purpose of the adjust- 
ment procedures was to modify each student's posttest score upward or down- 
ward sq as to eliminate or reduce v le presumed differential effects of the 
first set of factors, i,e., the pre-evaluation factors; in this way, theoretic- 
ally, the posttest scores would more accurately reflect the impact on achieve- 
ment of the student's learning experiences during the evaluation period. In 
the present case, determination of the appropriate adjusCments was based on 
the best-fitting regression of student-level posttest measures on pretest 
measures and student background variables. These equations were determined in 
the within-school regression space for each of the three evaluation samples 
(Pilot, Basic elementary, and Basic secondary), using the student as the unit 
of analysis* 

Table VIII-1 indicates the final set of variables used as covariates in each of 
the t^ree evaluation* samples. For all outcome measures, the preteist measure of 
that outcome was always the best predictor. For all outcomes except vpcabulary, 
the pretest on vocabulary (used as a generalized aptitude measure) contributed 
significantly to the prediction of outcome once prior achievement had been 
cTCcounted for. Finally, foi elementary students, the socioeconomic status (SES) 
measure derived from the number of educationally related possessions in the home 
contributed a small but meaningful amount to the prediction,^ The search was 
stopped at thic point, since the contributions of the highest priority variables 
were becoming quite small. 
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Because regression slopes departed noticeably from being homogeneous within the 
Basic samples, and because in these groups there was a substantial, proportion of 
non-minority students., separate regressions were determined as outlined above for 
minority and non-minority students. The set of predictor .variables for these 
separate regressions did not change, but the slopes did. Final analyses were 
performed using different coefficients for the two groups of students. Tests of 
equality of slppe within sample units were. satisfied when students were separated 
in this way. 

TABLE VIII-1. FINAL SET OF COVARIATES BY EVALUATION SAMPLE 



Evaluation Sample 


Covariates. 


Pretest 


Pretest 
Vocabulary 


SES/Readincr 


Pilot 


X 


X 


X 


Basic Elementary 


X 


X 


X 


Basic Secondary 


X 


X 





In attempting to establish a final set of adjusted posttest achievement measures, 
several steps were taken to deal with the fact that some students had missing 
data on the adjjistment variables. Since the number of students with data missing 
for race and SES/reading was small (less than 0.5%), an attempt was made to fit 
a value (i.e., to estimate the probable values of the missing data) rather than 
to drop such cases from the analysis. It was felt that further reduction of 
within-group sample size was potentially more damaging to the analysis than was 
the imprecision introduced by fitting these values. ' 

For secondary students, the primary data on racial identification were derived 
from the Secondary School Questionnaire. Where these data were missing, data 
from the Teacher Questionnaire were used. Where both types of data were missing ' 
for a particular student, the regression equation for the group (minority or 
non -minority) with the largest membership within' that particular, school and grade 
was used. 

For Basic elementary students, the Teacher Questionnaire provided the primary 
racial identification data, because it was. felt that this information would 
probably be more accurate than the children's own responses. When this was 
missing, the Elementary School Questionnaire data were used. Where data were 
missing from both sources, the same value-setting procedure used for secondary 
students was employed. For Pilot students (who by definition were in minority- 
isolated schools), a single regression formula was used, and racial identification 
was therefore not necessary. 

In the case of missing SES data, tlie within-school and within-grade mean value 
of the SES index for that student's own racial group was assigned to the 
individual. Where information on the student's race was missing, or where there 
were fewer tlian three students of that particular race in a grade in a school, 
the overall mean of students in that grade and school was used. 

The procedure described above allowed the prediction of an adjusted posttest 
score for every student having pretest and posttest scores. 
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a» Overall Impact Evaluation 



The primary means selected to evaluate the overall effect of ESAA funding was 
an analysis-of-variance framework. A treatment-by-blocks design was used in 
which the "treatment" was the funded/non-funded distinction, and districts were 
considered blocks. While the overall treatment effect could have been estimated 
by comparing the averaged outcome of funded schools and non-funded schools , the 
use of the treatment-by-blocks d'esign increased the precision of this estimation. 
This increase in precision occurred by taking into account the -known variability 
between districts. To the extent that the schools differed along dimensions 
associated with the outcome measures, the use of a treatment-by-blocks design 
would remove the contribution of between-districts variability from the error 
variance, thereby reducing the standard error of the estimate of the effect. 



b. Evaluation of Differential Program Impacts 

An important question in this evaluation was whether the impact of ESAA funding 
was different for different kinds of districts. To address this question, a 
series of analyses was undertaken to examine the differential impact of ESAA 
funding among defined subgroups of the districts. These analyses involved an 
extension of the basic design model to include a dimension along which districts 
were divided into subgroups (blocks) . Such dimensions could be defined for 
variables that had the district as a fundamental referent, or that were appro- 
priately aggregated to the level. of the district. 

The three types of blocking or suhrrr-ouping variables used in the evaluation of 
differential program impactjs are discassed along with the results of these 
analyses in a later section of this chapter. Briefly, the first type of blocking 
variable relates to program funding, the second to program* intensity , and the 
third to the focus of program objectives. It is important to keep in mind that, 
although the description of blocking variables refers to attributes of the treat- 
ment school, it was the district that administered the program; thus^ the variables 
discussed here reflect district-level decision-making. 



2. RESULTS 

The following sections describe the results of applying the methodological 
procedures described above to the data resulting from the first year of the ESAA 
evaluation. * 



a* Overall Impact ^ p 

1 ' 

Within the framework of the original experimental design, ^overall impact of ESAA 
funding was assessed by conducting treatment-by-districtrahalyses of* variance 
for each program type and grade level, using as the depehdent variable the 
covariate-adjusted posttest score aggregated to the school level. Separate 
analyses were conducted for each of the mathematics an^d reading subtests. As a 
check on possible anomalies resulting from the adjustment of posttest scores* 
all analyses were repeated using raw gain scores; in no instance was there a 
meaningful deviation in results for the two r^ethods. Thus, all analyses reported 
here are based on the use of the adjusted scores. 
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Table A-28 of the Appendix presents a sununary of each of the F values computcid. 
In each case, a si^gn has been attached to the value to indicate^whether the 
achievement test score difference favored the treatment school' (+ sign) or the 
control school (- sign) • / 

y 

/ 

Two analyses were conducted on the Basic elementary sample* One of these used 
all students; the other included only minority students so that* it could be 
compared with the minority-isolated Pilot sample* Results for both analyses 
are included in Table A-19, 

/ 

Overall/ the impact analysis didPnot show any ^neaningful/ interpretable patterns. 
The nun±)er of significant F values was no greater than would have been expected 
from sampling fluctuations alone. In sununalry/ there was no clear evidence .in 
this analysis of a differential achievemetit effect that might be attributed to 
funding. 



Differential Impact for Different Blocks of Districts 

m ^ 

Two sets of impact analyses were performed using blocking on district variables. 
The first set used variables related^ to funding levels in the schools/ while the 
second focused on two' general areas related to program characteristics. The 
definitions of the variables used and the results of each set of analyses are 
described in the following sections. Again/ all analyses were repeated using 
raw gain scores to verify tjjat no differences in outcomes resulted from use of 
the raw scores. 



(1) Blocking by Funding Levels 

(a) Definition of Blocking Variables * To assess the "overall- and differential 
effects of different amounts of regular/ supplemental, and ESAA funds on the 
success of instructional programs/ a series of blocked analyses were performed 
using funding data as blocking factors. The first factor was used to create 
three strata based on the amount of ESAA dollars available in the treatment 
school/ The second funding factor was based on the total supplemental funds in 
the treatment school (including ESAA) , The third factor examined the difference 
between treatment and control schools in terms of 'total supplemental fundS/ 
yielding categories where the treatment school got considerably more supplemental 
inoney than the paired control school/ about the same amount/ or less supplemental 
money than the control school, Th6 fourth factor used stratification of regular 
district funds, All^ funding figures were baseS on standardized per-pupil 
expenditures, • 

(b) Results of Impact Analysis Blocked by Funding . Tables A-20 through A-23 
ii> the Appendix summarize the F-ratios for the interaction between the effect of 
funding as a blocking factor and the treatfnent-control effect. For the three 
grade levels in each of the three evaluation samples, tests were made on both 
raw gains and adjusted scores on each of th3Slfour (or five/ at the secondary 
level) subtests. Thus, a total of 312 statistical tests were made in this 
analysis. Of these, only 16 (or 5,1%) were significant at the 5% level/ or no 
more than would have been expected by chance. Overall, therefore/ there was no 
clear evidence in these blocked analyses of treatment effect. 
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(2) Blocking by Program Characteristics* 



The impact analysis with blocking on program characteristics used tvo general 
program features as the blocking factors: program intensity and program focus. 
Each of these factors was separated into two content areas, reading and math. 
Program focus was operati9nalized by averaging the ranks assigned to various 
instructional objectives in reading and in math by the reading and math teachers 
in each school* 

(a) Definition of Blocking 'Variables ,, '^Jie variables used in the program 
intensity blocking analyses were derived from data related to student exposure 
to several reading and math activities. "fhe^Student Attendance and Exposure Log 
(SAL) provided a weekly record of the number of hours each ESAA student received 
instruction in reading,* math, and inter^roup activities* SAL data were collected 
on each student over a period of seven months, providing estimates of the stu- . 
dent's weeTcly exposure to program activities, ^he SAL exposure categories for 
each individual program activity were codefl to approximate an interval scale. 

^Summary indices for both reading and math exposure were computed by totaling the 
number of hours of instruction- in each component activity of the reading section 
and the math section, and averaging these total -figures over the number of months 
of data obtained for each student. These average total weekly e^stimates of math 
and reading exposure were aggregated to the school level for each grade in each 
of the threa evaluation samples. Trfeatment-control school differences in math 
and reading exposure were then computed, and grouped into two or more blocks for 
the blocking analyses. % 

(> 

The program focus variables defined the instructional program objectives repo?:tedly 
emphasized by teachers in thS four content areas of the achievement test, i.e., 
reading vocabulary, reading comprehension^, math computation, and^math concepts. • 

Two variables were used as indicators of reading program focus: ^ 

1. Word techniques—Emphasizing development of vocabulary skills through 
recognition of basic sight \vbrds, phonic analysis of words, and struc- 
tural analysis of w5rds. 

2. Word comprehension — Emphasizing devc^Ibpment of skills for using context 
clues, development of comprehension skills, and improvement of compre- 
hension rate. 

Two additional variables were used as indicators of math program focus: 

1. Math computation (emphasizing operations) . " 

2. Math concepts (emphasizing math concepts and terminology) .* - 



*Math ^ogreun focus indicators were badly skewed, since all teachers reported 
major emphasis on math concepts and operations. At the third- and fourth-grade 
levels in Pilot programs, insufficient variability in'*the proposed blocking 
variable precluded any attempt at conducting the blocked analyses. 
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Each individual reading and math program objective was ranked by the readirfg 
and math teachers as follows: major goal, "3"; secondary goal, "2"; little 
or no importance, **1." A high score on a particular focus index (e.g., word'^ 
techniques or math computation) indicated a strong emphasis on that objective 
in the teacher's reading or math program. The scores from the focus indices' 
at the teacher level were then taggregated to the scho^ level for each grade' 
in each of .the three ^valuation samples. ^ 

(b) Results of ' Impact Analyses Blocked by Program CharacTteristics > . For each of 
the six/' vai^iables described abcvo, a blocked impact analysis was con5«ctGd With ... 
botli raw gaiTi and adjusted scores on each of the four (or five, at the secondary 
level) subtests in the three grades in each of the three samples. Tables A-24 
through A-2$ in the Appendix present the JPnratios for the interaction effect 
between the blocking factor and the treatment-control effect., A total of 452 
such tests were made, of which only 22 (or 4.86%) were significant at the 5% 
level. Few of the significant interaction effects showed any meaningful pattern, 
and none showed consistency across grades pwithiri subtests pr across sxobtests 
within grades. - ^ ' . * ' 

/ > • > f 

4 . SUMMARY . i ^ • ' ^ / < • 

As noted at the beginning of this chapter.- short pretest-posttest -interval, 

combined with the relatively small treatment-control difference? in f\inding levels 
and progifain characteristics, made it improbable that consistent* or sizable 
differences in achievement test gains would be found* It was not surprising, 
therefore, that the impact analyses showed no .evidence of significant treatment- 
control differences in achievement gains. This finding was consistent across 
grade levels, criterion measures (reading and math)", and program types ^( Basic 
atid Pilot); furthermore, no" treatment-control differences were found when 
districts were subgrouped (blocked) on several dimensions related. to funding 
le^pls, program intensity, and program -objectives . 

■ - V • . . '. ' 

B. ANALYSIS OF TE ST GAINS IN hbAA-ELIGIBLE SCHOOLS ^TREATMENT PLUS CONTOOL) 
* : — ■ ' " " — ■ ■ 

n 

An important issue in this study was whether student^ 'in the evaluation sample 
ro«dp achievement test gains during ^the school year th^t were of statistical aTid/. 
practical significance. To examine* this question, analyses of achievement gains 
were performed across both treatment and control schools. 'One argument -for 
including the control schools was the finding, reported dn earlier chapters, that 
treatment and control schools in many districts did not differ greatly in either 
per-pupil funding or program cha>-acteristics. In addition, informal discussions 
with school officials indicated that, in certain distridts at least:, the award 
of ESAA funds to those districts might hav6 led to greater total per-pupil 
allocations in control as well as treatment schoals, due to shifts of non-ESAA 
supplemental funds to the control schools. For these i?easons, emphasis in the 
present analyses was placed on academic growth in the sample of ESAA- eligible 
schools, as represented J^y both treatment an^ control schools. These ESAA- : 
eligible schools typically received greater supplemental funds ot varibus types 
(Title I, Title III, etc.) than non-eligible schools because thf>y have larger 
proportions of disadvantaged students. Analyses of overall achievement gains 
across the eligible schools may therefore help to assess the possible beneficial 
^iffects of the total supplemental funding. 
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1. STUDENT ACHIEVEMENT GAINS AGGREGATED ACROSS DISTRICTS 

Some gains in absolute performance between pretest and posttest would, bf 
course, be expected irrespective of supplemental school^ ftinding. The question 
is whether these gains were greater than might have been expected in the 
sample schools had those schools not been receiving such extensive supplemental 
funding. One set of/benchmarks for assessing the sample schools* gains can be 
obtained from teat publishers* normative tables • By using thase tables to cal-^ 
culate grade equivalents for the sample students"* pretest and posttest scores, 
amounts of actual gain in grade ^equivalent CGE) units can be determined. Table 
VIII-2 shows the pretest and posttest grade equivalents for achievement levels 
of the sample students used in all subsequent analyses •* For purposes of these 
analyses, all student scores within a given grade and program type were combined, 
irrespective of treatment-control designations. To illustrate the interpretation 
,of the table values. Pilot third-graders, who should ^by national normative 
standards have had a grade equivalent of 3.10 in reading vocabulary at pretest, 
actually had a grade -equivalent of 2*10, ,which corresponds to the perforrftance 
expected of average students at the beginning of the second grade* Similarly, 
the expected grade equivalents for average studeints at the time, the, Pilot third-, 
graders took the posttest would be 3.80; the actual grade equivalent in reading 
vocaJDulary^ f or those sample Pilot students was 2.65. 

The final colxarm in Table VIII-2 shows the difference, in grade-equivalent units, 
between the sanple students* pretest and posttest achievement scores. Since the 
average interval for sample students was very slightly over five-and-a-half months 
(168 days), it might have been expected that average students would, overall, 
have made test gains corresponding to .55 grade-equivalent \initsl As seen in 
Table 7111—2, the actual gains in grade equivalent approximated or exceeded 
this "average" gain figure foremast grade levels and most* subtests across both 
the Basic and Pilot samples. This appears to be a very positive finding in view 
of the fact that the sample students were in no sense "average" students. Rathe?, 
the/ were <=*<^oncn*.icaliy disadvantaged and had a prior history of making considerably 
smaller-than-average achievement gains, as evidenced by their low grade- equivalent 
scores on the pretest. Thus, it would not have been expected that the sample ' 
students would have made pre test-post test ga^ns as large as those of average 
students. 

Ix3oking at particular programs and grade levels in Table VIII-2, Pilot elementary 
students" (treatment and control) made gains at or slightly below those expected 
for average students in reading and generally above the expected** levels in 
mathematics. Basic elementary student gains were generally around the expected 
values for reading and well above those values in reading vocabul^^ry at the 



*These "^rade equivalent's have been interpolated to half-month intervals to 
account for cases where the observed mean fell almost exactly between the 
scores associated with the full-month values, 

**For simplicity, the term "expected gains" is used throughout this section when 
referring to the gains expected of average students* It should be recognized/ 
however, that for reason^ discussed in the preceeding paragraph, such gains 
would not have been expected of the sample students. 
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TABLE VIII-2. STUDENT PERFORMANCE LEVELS (TREATMENT' AND CONTROL) 

AND GRADE ' EQUIVALENTS p 



Reading Vocabulary 





Pretest 


Posttest 


GE 


Program Grade 


GE* 


GE- 


Difference 


Pilot 3 


2.10 


2.65 


.55 




2.40 


2.90 




S * 


3.20 


3.75 


•-55 


Basic 3 


2.60 


3.10 




4 


3. 00 


3.80 


.80 


5 


4.00 


4.75 


.75 


Reading Comprehension 




Pilot 3 


2.20 


2.70 


.50 




2.70 


3.20 


.50 


5 ' 


3.40 


3.80 


.40 


Basic 3 


2.70 . 


3.00 


.40 ^ 


4 


3.20 


3.75 - 


.'55 


5 


4.10 


4.60 


.50 


Mathematics Concepts 


Pilot ^ ' 3 


1.80 . 


2.50. 


.70 


4 


2.30' 


2.90* 


.60 


5 


3.10 


3.70 


.60 


Basic 3 


' 2.30 


2.90 


.60 


4 


2.80 


3.60 


.80 


5 


4.30 ' 


4.90 


.60 


10 


8.10 


8.40 


.30 


11 


8.40 


8.90 


.50 


12' 


8.60 


9. GO 


.40 


/ Mathematics Computation . 


{ 

Pilot 3 


2.20 


3.00 


.80 


4 


3.10 


3.70 


.60 


5 


4.00 


4.40 


.40 ■ 


Basic 3 


2. '40 


3.30 


.90 


4 


3.30 


4.00 


■ .70 


5 


4.30 


4.90 


.60 


10 


7.60 


8.10 


.50 


11 


8.20 


8.60 


.40 


12* 


8.40 


8.90 


.50 - 


*Grade equivalent 
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TABLE VIII-2, STUDENT PERFORMANCE LEVELS (TREATMENT AND CONTROL) 
AND GRADE EQUIVALENTS (CONTINUED) 



Mathematics Problems 



Program 


Grade 




Pretest 
GE* 


Posttest 
GE 


GE 

Difference 


Basic 


10 




7,60 


8.30 


• 70 




11 




8,10 


9.00 


,90 ' 




12 




8,60 


9.30 


,70 



*Grade equivalent 



< 

fourth and fifth grades; in math the gains were almost uniformly well above the 
expected levels. Gains of the Basic secondary sample students ran somewhat 
beiow the expected values in math concepts and compiitation, but consistently, ^ 
and by a coosiderable margin, exceeded expectations in math j)roblems. 

It is iirportant that cohsid^able caution be exercised in interpreting these 
results, because the publisher norms on which the comparisons are based were 
interpolated for other than end-of-year performance. In constructing the norms, 
the publishers tested samples of students late^ in the school year at each grade 
level. Assuming that no growth wpuld occur over the summer months and that 
growth over the period of schooling^ would be linear, they interpolated norms 
for beginning and middle-of-year testing periods. The validity of these assump- 
tions has not been tested, nor is is readily possible to obtain data collected 
at various periods over the school year and suihmer on comparable samples of 
students. • ' * 

A second concern is the shape of the growth curve typically associated with 
standardized achievement tests during the school year. Is growth linear during 
tlie school year, or i^ it 'more rapid at certain periods than at others? No 
available data provide a clear answer to this question, yet it is an important 
consideration, in attempting to make statements about "expected" growth duri: g 
some segment of the school year. 

Finally, even assuming that gains of the sample students were higher than might 
have been expected, there is no basis for attributing the gains to the effects 
of ESAA dollars, or even necessarily to those of supplemental funds in general. 

Despite these reservations, there appear to be important indications in the 
first-year evaluation data that disadvantaged students in the ESAA-eligible 
schools made achievement test gains very similar to those expected for average 
students. Such a fiTftding, however tentative, is quite encouraging and will 
be further explored in the analyses of second- and third-year data. 



2. COMPARISONS OF SCHOOL-LgJ/EL GAINS WITH GROWTH CRITERIA 

Whereas the preceding section aggregated achievement data across districts, the 
analyses reported in this section use achievement data aggregated to the school 
level. For purposes of this discussion, the publishers' test norms and the 
assumptions about s.tudent growth (e.g., interpolation of norms for non-einpirical 
testing points,; assumption of linear growth) are accepted^ It must again be 
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recognized, hpwever that these are assumptions and are subject 'to later verifi- 
cation, to the extent possible. The longitudinal nature of the evaluation 
allows- later testing of the no-growth-over-summer-months assumption implied in 
tfte interpolation procedxares used by the test publishers.. Exainii^a|:ion of 1975 
pretest data will be crucial in the final assessment of first-year gains. 

Analyses discussed in this section a'gain address the question of whether gains 
in the sample schools (treatment and control) were larger thcin could have been 
expected 'in schools having large concentrajbions of disadvantaged and academically 
needy students. One difficulty in answering this question is that ther^ is nc> 
established consensus among -educators or psychometriciarls on the definition of 
."expected" gains against which observed gains shquld be compared. In particular, 
there is no general agreement as to how^allowance should be made in these 
expectations for the economic cind educational deficits of the sample students. 
^To provide a range of possible interpretations and a framework- within which 
readers can reference their own expecta\:ions, three different growth criteria 
were established for the present analyses. . These are described in the following 
paragraphs . 

Criterion 1; Growth at the median level for the normative sample , " For each 
grade level, a calculation was made of the Achievement Development Scale Score 
(ADSS)* associated with the median (50th percentile) of' the pretest and posttest 
distributions. On the assumption that these values represented a typical stu-r 
dent' in the publishers' normative sample, a determination was made of how much 
change between these two scores was necessary for the student beginning at the 
50th percentile to maintain that same relative position at posttest. Such a 
calculation ignores certain problems of metric properties of, changes at a 
particular score level,** but is useful for comparative purposes vis a vis 
"typical" growth, 

Crite^rion 2; Maintenance of pretest percentile . For each of the evaluation 
samples, at each grade level and for .each subtest, the percentile rank associated 
with th6i^ observed pretest mean score was obtained from the publishers" norm 
tables, ^he scores necessary J:o maintain this same relative position in the 
posttest di^GtriDution were obtained from the same tables. The difference between 
these two scores was then used as the second criterion for the particular grade 
level and subtest in question. Depending o,n whether the pretest and posttest 
distributions retain the sam^e variabilities or change" in variability, this 
criterion value can be quite different from that generated by the first criterion. 



*A single scale of standard scores for use with .all levels and any form of the 
CAT has been developed. These ADSS scores use three-digit numbers ranging 
from 100 to 900. They enable the user to chart a student's growth from grade 
1.5 through grade 12, regardless of which level or form of the CAT is used. 

**Such problems fiiclude the difficulty in raising achievement scores by one point 
for low achievers (many remaining items, of a wide difficulty range) versus 
effecting a score difference of one point for high achievers (few items remain- 
ing-, all of which are very difficult) • 
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Criterion 3; Depressed normal growth rate > There is some tentative evidence 
from earlier studies of compensatory education, such as local^ ESEA Title I reports, 
that disadvantaged students such as those in the present study typically make 
achievement test gains approximately two-thirds the size of those made hy average 
students as defined by national test norpis.* Though this presumed relationship ' 
has not been ^iven enough systematic study to be well established, it is certainly 
consistent with tKe tertdency of- disadvantaged students to fall further and 
further behind^ their more fortunate agemates as they move through successive 
grades. ' 

For elementary, students, data from SDC*s earlier rest^ndardization study .were 
^ used 'to estimate the growth rates of disadvantaged students.** The restandard- 
ization data provided grade-equivalent values of actual scores for Pilot-eligible 
students at the end of the third,, fourth, and fifth grades. These enpirical ^ 
values were compared with expected values, based on the stud^rrTs^' grade level and 
time of testing, to determine the restandardization sample's relative growth rate. 
For example, a third-grade student tested in May. should have a grade, equivalent 
of 3.8; if his actual grade equivalent was 2.9, this wou^d mean Lliat he was pro- 
gressing art: about three-fourths the average rate for the third-grade population 
as a whole. Actually, it was found that the growth rate values for tlxe restan- 
dardiza.tion sample, across grades 'and content areas, ranged in a very narrow band 
around the popularly accepted two-thirds value. To establish the third criterion, 
therefore/ these empirically developed growth-rate t?alues were used as scale 
factors in a downward adjustment of the gains required tq^meet Criterion 1. For 
example, if the criterion of maintaining median position (Criterion 1) involved 
a pretest-posttest gain in grade equivalent of 0.6, but the previous gain rate 
^ of a particular sample group was only seven-tentJis of the gain for average 
students, then the "expected" gain for that stibsample would be adjusted to 0.6 x 
0.7, or 0.42 grade-equivalent units. . . 

At the secondary level. Where empirical baseline data on growth rate^ were not 
available, the two-thirds approximation was used as the scaling factor. 

Since all analyses were conducted using scale scores ^ several comparisons and 
interpolations were necessary to obtain the required scale-score differences. 
The publishers' norms provide percentile ranks associated with discrete raw 
scores. When no discrete score fell at the 50th percentile point (or at the , 
percentile point associatpH with the pretest criterion), interpolations were 
made. Once determined, these raw-score points were converted to ADSS score 
points, again often requiring an interpolation. All interpolations were linear 
■•over the score interval, and high accuracy was maintained. 

It is extremely important in interpreting these criteria, and in assessing first- 
year achievement gains, to observe the necessary cautions implied in the con- 
struction 'of the criteria. The metric properties of these score differences, 
which are based on interpolated data and anchored to particular score points in 
the distribution, ,are wholly unknown. The criterion-related difference values 
should be viewed simply a? crude indicators of amount of expected growth. Any 



*For a summary of findings of some of the earlier studies see Wargo, M.J., 
Compeau, P.L.,, and Tallmadge/G.K. Further Examination of Exemplary Programs 
for Educating Disadvantaged Children , American Institute for Research. 

**02enne, D.G., Van Gelder, N.C., and Cohen, A.J. Emergency School Aid Act 
National Evaluation; Achievement Test Restandardization , TM- 52 36/006/00, 
System Development Corporation, November 1, 1974. 
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confidence in such findings must depend on consistency of patterns over different' 
grade levels and over related subtest measures, rather than on any presumed^ 
precision in a particular "expected** growth value. Thus, one^itay be able to 
obtain a reasonable answer to the question, *^id a large number of schools 
exhibit average performance greater than the criterion?", but not to the 
question, "Did exactly 75% of the schools exhibit precisely one year'-s worth of 
growth during a year of schooling?'* - > 



1. ANALYTIC TECHNIQUE ~ 

Once the effectiveness criteria .were established, one serious, methodological issue 
remained in attempting to evaluate the, success of individual programs (or 
groups of programs) in meeting the criteria. This was the issue of spurious 
intergroup differences when multiple .con5>arisons were pade. In, addition, it was 
desirable to make some statement of the confidence one might have in the observed 
gain. , ^ ^ 

The approach in analyzing these data derives from the Bayesian methodology. 
The specific methods used in the present analysis were provided by Novick, 
Wang, and Isaacs (1975). These techniques are especially helpful in making 
inferences about the success of a particular compensatory program when many 
individual progca^ comparisons are being made, i.,e., in solving what, the research 
methodology literature calls the "multiple comparisons problem." Briefly, the 
multiple comparisons problem arises when it is desired to compare observed 
performance against: performance for a large, number of programs and when the 
number of observations for any one comparison, is relatively s^all. In such 
situations, estimates of these individual program effects are imprecise, and 
apparent differences -between programs (schools) may be found when those differences 
have occurred by chance and do not. represent actual program effects. The Bayesian 
techniques employed here, termed M-group analysis, substantially increase the 
precision of the estimates because they use not only the direct information 
obtained from the observations in that particular unit, (e.g., in a grade level ^ 
within a given school within a program type) , but also collateral information 
(performance of grade- level cohorts in other schools in the same program) . The 
net effect of the Bayesian approach is a "pushback" of thje individual estimates 
toward the overall mean in a manner analogous to using regression estimates of 
true score in place of observed scores in classical test the'Ory. The Bayesian 
teqhhiques provide estimates of average growth and associated variances as .y^ell 
as estimates of individual program effects. • • 

> 

The model for this analysis assumes chat for each student there is a pretest ahd 
posttest. The means for a school on these two test administrations can be 
denoted as \iB and PA, respectively. The primary concern is with the difference 
A = ]jA - pB, the a^verage difference between posttest and pretest scores* If this " 
value is larger with respect to some criteripn Aq, then it may be concluded that 
some gain has occurred. The true value of A cannot be known, but; by using a 
Bayesian analysis,, prior information can be combined with sample information to 
provide a posterior probability distribution for A. Then the a posteriori prob- 
ability that A > (that the gain is greater than <2xpected) is computed. If 
this probability is greater than .50, one might cautiously suggest that in- 
struction has been effective in this school. If the probability is greater 
than .90, then one can conclude with much greater confidence that instruction is 
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moving children, on the average, past the growth interval required ,to meet 
the criterion. 



2. RESULTS 

Using the methodology described above, gains v>?ere analyzed for each school in 
treatmentrcontrol pairs, for each evaluation sample and grade level. The tables 
in this section describe the percentage of schools in each of these groups whose 
mean achievement gain met or exceeded a given criterion at the stated probability 
level. The column entitled "Maintaining SO'th Percentile" in Table VIII-3 in- 
.dicates the actual percentage of schools whose average gain was equal to or 
greater than a ^particular criterion. The second column indicates the percentage 
of schools exceeding the criterion wi-th probability qreater than .90. The .90 
level provides strong evidence that such a result reflects a true and stable 
effect rather than a chance occurrence. 



a. Pilot Elementary SchooJ-s 

In readincj y the percentage of Pilot elementary schools having pbserved growth 
(test score gains) that met or exceeded the. "disadvantaged student" criterion 
(Criterion 3) ranged from 43% to 81%, depending on grade and subtest. F6r five 
out of six grade-subtest combinations, over half the schools exceeded that ^, 
criterion. The percentages of total Pilot Sample schools meeting or exceeding 
Criterion 3 at the .90 level of confidence ranged from 13% to 47%; these 
, correspond to roughly 30% to 60% of the schools observed to meet the criterion • 
Somewhat fewer schools (37% to 75%) were observed to maintain their relative 
pretest position (Criterion 2),.but^over the various grade-svibtest combinations, 
more than half of the schools met that criterion; this in itself may be an \ 
encouraging sign considering the general tendency of ^disadvantaged students to 
lose ground in successive, school years. 

Within the reading area. Pilot sample students (treatment and ^control) performed 
slightly better in comprehension than in vocabulary, but results were, quite 
varied at different grade levels. Overall, reading performance was highest at 
the fourth-grade level; that is, a larger percentage of schools met or exceeded 
each growth criterion at this grade than at either the third or fifth grades. 

. fn mathematics. Pilot schools demonstrated much greater relative growth than ^ 
in reading. This is reflected in the larger numbers of schools (well over half) 
meeting each criterion for almost every combination of subtest and grade level, 
and in the increased number of schools meeting the criteria at a .9J) level of 
confidence. Performance was particularly notable in math concepts, where from 
79% (grade 5) to 97% (grade 3) of the schools were observed to meet the "dis- 
^advantaged student" criterion (Criterion 3), c\nd. from 50% to 80% maintained 
relative pretest position (Criterion 2) . The percentages of the total Pilot 
sample schools meeting or exceeding Criterion 5 at the .90 level of confidence 
in math concepts ranged from 47% to 83%; these correspond to roughly 60% to 
85% of the schools observed to meet the criterion. Furthermore, the results 
were almost as encouraging for Criterion 2 (maintaining position) , particularly 
in math comprehension. Overall, the evidence suggests strongly that Pilot 
elementary students made gains in math considerably larger than would normally 
have been expected of disadvantaged students. 
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The percentage of schools exceeding the various criteria in math was higher 
at the lower grade levels than at the higher grades. This may indicate that 
the school programs were relatively more effective for younger students — a 
possibility that will be investigated more closely in subsequent reports using * 
longitudinal data. 



b. Basic Blementary Schools 

In Basic elementary schoqls, reading gains were' generally similar to those for 
Pilot schools, with 44% to 99% of the schools observed to meet or exceed the 
"disadvantaged student!! criterion, and 26% to 9,4% maintaining relative pretest 
position (Criterion 2) . Overall, well over half the schools met the "disadvantaged 
student'^ criterion; of the schools observed to meet the' criterion, a .90 confi- 
dence level can be placed on roughly 55%.* 'As in t±e Pilot sample, gains were 
higher in the fourth grade than in the third or the fifth. 

Mathematics gains in Basic elementary sample schools exceeded each of the 
criteria in large percentages of schools. For example, the percentages of 
schools obseifVed to meet or exceed the "disadvantaged student*' criterion ranged 
from 50% to 100%, and the corresponding boundaries for Criterion 2 (maintain 
pretest 'position) were 41% to 94%. A very positive finding is that even the 
most stringent criterion (gain equivalent to maintaining a median position) was 
met by 41% to 90% of the Basic elementary Schools; more than half of these observed 
cases meet the .90 confidence level. 



c. Basic Secondary Schools ^ 

Only mathematics subtest results are available fo^r Basic secondary schools, 
because no growth criteria could be dexrived from the publishers ' norms in 
reading that would be analogous to the other criteria used here. It should also 
be noted in Table, VIII-3 that there wa6 an additional math subtest at the 
secondary level ( "problems'' ; that was not given at the elementary level. 

Mathematics gains at the secondary level were highly encouraging, with sub- 
stantial gains indicated by the numbers of schools meeting or exceeding each 
growth criterion, aqross subtests ^nd grade levels. Percentages of schools 
observed to meet the "disadvantaged s^tudent" criterion ranged from 50% to 82%; 
corresponding ranges for Criterion 2 (maintaining relative pretest position) 
and Criterion 1 (gain ^uivalent to maintaining a median position) were 41% 
to 90%, and 41% to 82%, respectively. 

Unlike the elementary schools, average gains in the secondary sample were 
higher in^ the higher grades. One possible explanation for this finding is 
selective attrition; that is, the poorer-achieving students n\ay have dropped 
out of school. Longitudinal data in subsequent years of the study will be 
examined in an attempt to verify or refute this explanation. 

C. SUMMARY OF GAIN ANALYSIS RESULTS ' ' 

Treatment-control comparisons were not expected to show appreciable differences 
in achievement-test gains for the two groups, because of the short pretest- 
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posttest interval, the early developmental stage of the ESAA programs, and 
the lack of major differences in many of the treatment-control pairs with 
respect to total per-pupil funding and program characteristics. Nevertheless, 
the analyses were performed, using analysis-of- variance techniques with and - 
without blocking on certain district characteristics; the results, *as antici- 
pated, did not show any consistent pattern of treatment-control outcome 
differences. 

A somewhat more positive picture was presented by analyses using achievement 
test data for combined treatment and control schools in the sample ESAA districts. 
In these analyses., it was fotond that many of the ESAA-eligible schools (both 
treatment and control) made average achievement test gains larger than might 
have been expected of disadvantaged students; in a fair number of cases they 
made gains as large as would have been expected of "average" students (based 
on national norms). These results appear highly encouraging in view of the 
fact that disadvantaged students, such as those represented in this, evaluation, 
traditionally make substantially smaller gains than those of average students. 
Considerable care must be taken in interpreting the results, however, since 
the analyses involve a number of unsubstantiated assumptibns concerning the- shape 
of growth curves in the general population during a school year. Furthermore, 
the results were not entirely consistent or vmifoimly positive. Finally, even 
if the achievement gains were larger than would normally have been expected, 
there is no basis for attributing such results to ESAA funding. Although it is 
suspected thatj the relatively large amounts of total supplemental funds found 
in most ESAA-dligible schools might account for enhanced gains in the sample 
schools, no direct evidence is presently available to support such a contention. 
Analyses of second- and third-year data will be important in attempts *to confirm 
some of the apparent trends and relationships observed or inferred in the initial 
evaluation year. ^ ' » . 



134 

VIII-18 



I 



CHAPTER IX 



EXPLORATORY ANALYSES 



This chapter describes a set of exploratory analyses of the dSta collected in 
the first year of the ESAA evaluation. These analyses were non-experimentai 
in nature; that is, they were>not based on comparisons of randomly assigned 
treatment and control, schools within districts, but instead cut across both 
categories of schools. Their purpose was to determine what types of school . 
programs (regardless of funding source) were most effective for what types 
of students. The exploratory analyses contrast with the more general descrip- 
tive analyses reported earlier in this document (e.g.. Chapters III through 
VII) , in that the exploratory procedures usually, involved more complex 
relationships among student, program, and outcome measures. They also differ 
from the experimentally- based impact analyses (Chapter VIII) , in that the 
exploratory procedures examined ci wider range of possible relationships, but 
at some cost in generalizability of the findings. Details of the exploratory 
analyses are presented and results are discussed in later sections of this 
chapter . 

Within the general goal of defining relationships among student, program, and 
outcome dimensions, three objectives of the exploratory analyses can be dis- 
tinguished. The first objective stemmed directly from the requirement 
expressed in the study's original Statement of Work, to "examine the inter- 
relationships between program input, process, and output variables to determine 
which variables can be manipulated to improve program impact," The second 
objective, closely related to the first, was to explicate or elaborate the 
results of the school-level analyses of achievement gains, ,The third objective 
of the exploratory analyses was to determine what interactions and non-linearities 
e^^sted in' the relationships of the inpuL, process, and output variables examined 
in^ this study, and how those interactions affected the substantive interpreta- 
tions of the study's results. 

Before the exploratory analyses could be conducted, several preliminary *steps 
were required to determine the most useful subset of data elements (questioh- 
naire items) to be used in the analyses, Thes^ steps involved careful screening 
of elements from the 3000-plus items in the original data set, so as to retain 
those items that would be most useful and relevant to the study objectives. 

Efforts were made to select items that would (1) be readily interpretable for 
policy decisions and (2) reflect manipulable sc^iool, program, and process 
variables that might be expected to have direct impact on student outcomes. 
In many cases / two or more related items were combined into new composite 
Vc\ria^)les, so as to help reduce the total number of items to fewer and more^ 
interpretable dimensions, 
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Two basic procedures were used in selecting items and-^n forming con^posite 
variables for use in the exploratory analyses. One approach, used in the 
early screening stages, involved a rational examination of all items to 
determine their relevance to the major evaluation issues and to ascertain' 
which groups of items appeared to be tapping the same underlying dimensions 
or variables. Results of .earlier studies were also used in this phase to 
pinpoint variables that were most likely, to show significant relationships 
t6 student achievejaent. The second general approach was empirical, and 
involved preliminary correlational and factorial analyses tO' determine which 
items appeared to be related to each other and to achievement. By these 
combined procedures, the original set of items was reduced to, a set numbering 
fewer than 150 items and composites. 

The next section of this chapter describes the classes of items and composites 
retained for the explorat^ory analyses; Following this is a brief discussion 
of the statistical procedures used to perform those analyses. Finally, the 
results of the exploratory analyses are described and interpreted. 

A. VARIABLE DEFINITION 

. - r - 

This section describes the (independent and dependent variables that were 
utilized in the exploratory arialyses. Specific methodological and substantive 
issues in the definition- of the dependent variables (achievement gains in 
reading and math) are discussed separately. 



1. INDEPENDENT VARIABLES - 

One of the major tasks in the exploratory analyses was to specify, from the 
data available, a constellation of variables that might potentially influence 
academic achievement. The categories of independent variables selected and 
the general' methodology used to define the variables are described below." 

a. Categories of Variables 

categories of independent variables* were designated as follows: 

m Individual student characteristics 

• Individual teacher' characteristics 

• Instructional program (classroom) characteristics 

• Contextual (school and district) characteristics * 

• Funding allocations 

« 

(1) Individual Student Characteristics 

Among variables expected to have the most direct effect on learning were several 
characteristics of the student h"»tnsplf . These included two major sociodemo- 
graphic characteristics: ethnicity and socioeconomic status. The student's 



*The final set of variable mnemonics and their identification is available 
in Appendix A, Table A-39. 
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academic orientation (e.g., his liking foj?^school, his feelings about the 
importance of good grades, and his personal education expectations) was also 
examined, as were certain psychosocial characteristics such as internal/ 
fexte^rnal locus-of-control and the nature of the student's interacti6n with 
Ms peers. Finally, the teacher's educational expectation of the student 
was treated, for purposes of the present analyses, as a student characteristic. 

✓ 

(2) Individual Teacher Characteristics 

Selected characteristics of the student's teacher defined a second set of 
independent variables. These included, for example, the teacher's general 
training and teaching experience, hi^ specialized inservice training in 
reading or math, and indicators of his effort and style, such' as the number 
of .Resources generated by the teacher to supplement instruction and the 
flexibiliJ:y-or rigidity of his teaching approach. ' - 

(3) Instructional Program (Classroom) Characteristics 

A third category of variables included certain characteristics of the instruc- 
tional programs used in reading and mathematics, many of whicK^were described 
in Chapter' VI. Potentially important variables were the actual amount of time' 
spent in reading and math activities (estimated by average* total weekly 
exposure in reading and ma.th) and the* allocation of time spent in individual 
jheading and math activities Another set of variables examined here related 
to program focus, defined in terms of amount of emphasis on particular activities 
in reading and mathematics. ; 

./4) Contextual (Sphool .and District) -"Chairacteristics 

The broader' content of the student's le'kming environment was defined by 
school- and district-level variables, Sociodemographic characteristics examined 
here Iftcluded minority percentage in the school and overall student body SES 
level, AS indicated l?y the principal's estimate of the percentag^i of students 
eliqlble fox, Title '^I funds. Also of interest -were the teacher/pupil ratio in* 
the school and the number of specialized readin^^nd math personnel relative 
to total staff si;2e.>- ^ y 

Other contextual variables used in t^ie exploratory analyses were characteristics 
related to desfeigreqation programs and racial climate of the schools, such as 
the extent of bussing in the district and Nthe extent of disruptions having 
racial/ethnic p^cigins. IT^e degree to which districts promoted desegregation 
programs was indicated by^ a frequency cQuiit of district activities such as 
media presentations, public meetings, and staff meetings. 

Funding , variables were a particularly impoirtant aspect of the district/school 
context, since they comprised a major set of variables in the evaluation design. 
School-level fundirtg variables were defin^ed in terms of total supplemental 
dollars per pupil spent in reading and math. District cost data were used to 
define regular dollars-per-pupil expenditure variables. 



.b. General Methodology Used to Define Variables 

Sihce.the unit of analysis in measuring academic 'achievement was the student, 
all data on independent variables, including data collected at the school and 
district level, were dis-dggregated, where necessary, to the individual stu- 
dent level. The complete specifications of these variables, including their 
questionnaire components, deriva^tion, and coding, will be made available in a 
separate document. Since the questionnaire data were largely categorical, and 
at best ordinal in nature, single items were either coded to define dummy 
variables or ranked. Multiple-Item composites were expressed as proportions, 
as net frequency counts of Ci^^tain item responses, or as average ranks or rank 
sums of items representing a dimension or logical set. 

2. DEPENDENT VARIABLES , 

The residualized. total scale scores from the reading and mathematics achieve- 
" ment tests were the dependent variables for the overall exploratory analyses. 
The derivation and the 'rationale for the use of these scores are presented 
in the fallowing paragraphs. ' 

At the elementary level, the reading raw score was obtained for performance 
on the entire reading section ^of the California Achievement Test, Level 2 
(grade 3) and Level 3 (grades ^4 and 5). * At grade 3, the mathematics test was 
composed of the items from the computation and concepts subtests.. Level 2, 
omitting the problems section of the latter sxabtest. For grades 4 and 5, the 
mathematics- test Consisted of items from the computation subtest and the concepts 
subtest. Level 3, omitting the 20 items dealing with fractions in €he computation 
subtest. ' J 

At the secondary level, the reading scjdifes were based on performance on the 
Iowa Silent Reading Test, Level 2. The reading raw score was a total of raw 
scores on the Vocabulary and Reading Comprehension subtests. The entire 
mathematics test of the California Achievement , Test , Level 5, was^ used to 
obtain the mathematics scojre utzilized in these analyses. 

The total raw scores v/ere converted 'to total scale scores employing the nc^rm 
tables furnished by the test publisher. This conversion allows direct 
comparison of results across test forms -and test levels. The total score 
was used rather than subtest scores because it required fewer analyses 
while still providing adequate information. " , 

The decision to use residual scores in the regression equations was based on 
preliminary empirical investigations indicating that the pretest scores would 
account for most of the variance in the posttest scores. It was clear that 
the pretest would be the best single predictor in the regression equations, ^ 
yet its inclusion would severely limits the interpretability and usefulness of 
analytic results. In particular, it was 'feared that the strength of the 
pretest variable^ght mask the effects 6f -other possible predictor variables. 
' Thus, the decision v;as made to account for pretest effects by the use of 
residual scores, and to concentrate in the regression equation itself on 
manipulable characteristics. 

Regression analysis was used to calculate the residual scores. Two regression 
analyses were performed at each grade level, one for each dependent variable 
(reading and math) . The subtest scale scores from the achievement pretests 
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were used as predictor variables, and the posttest total scale scores were 
used as dependent variables. The predicted posttest* total scale scores 
resulting from these regressions were then subtracted from the corresponding 
actual posttest scores to produce the posttest residual scores. The residuals 
produced by this procedure can be considered to have had the effect of the^ 
pretest score levels totally removed. In the regression analyses to be 
reported, it should be remembered that only residual variance is being / ^ 

accounted, for, and results should be interpreted accoi;"d^ngly . 

B, methodology' 

The primairi^ statistical method employed in the overall exploratory a'nalyses was 
linear multiple regression. Multiple regression teciiniques were chosen as the 
primary tool of the exploratory analyses because of their i:raditional role in 
exploratory analyses, and because o^ the straightforward interpretability of 
their results. Specifically, the results reported in this chapt:er were obtained 
by means of a stepwise multiple regression procedure designed to build the final 
regression equation through a series of steps, each of which adds one independent 
variable to the^quation. The independent variable ad^ed at each step of the 
procedure is the one which contributes most to the predictability of the 
equation. The advantage" the stepwise regression is that it yields a good 
approximation of the "best" set of predictors. 

Several cautions concei?r]|^ing the use and interpretation of stepwise multiple 
regression should be mentioned. One assumption of the multiple regression 
model is perfect measurement in the predictor variables, i.e,, that no error 
of measurement exists in the data. When this condition is not met, the 
calculated coefficients tend to be underestimates of the parameter values; that 
is, the apparent contribution of a predictor variable to variance in the outcome 
measurement may be less than its .^w^contribution.- In the present case, since 
error 'of measurement definitely exists, the obtained regression coefficients ^ 
are conservative estimates of the true values. 

A second factor that may have an effect on the calculated results is^the way 
in which mJLssing data were handled for these analyses. The mean value of the 
total sample was substituted for the missing values for those variables when 
the mean was an intetpreta'ble statistic. This procedure tends to further 
reduce the magnitude of the calculated regression coefficients/ and makes the 
estimation procedures even more conservative. 

* ■ 
Finally, the multiple regression technique. used here was a linear model, and 
such a r^iodel can explain only linear relationships. While linear relationships 
are extremely useful in suihmariziYig data, and the largest component of most 
behaviorc\l relationships is usually linear, this by no means excludes other 
components. * In fact, there is substantial evidence that curvilinear relation- 
ships exist between the predictor variables and the criterion variable in the 
present data. The effect of this curvi linearity is to lower the precision of 
the obtained regression results. Mor^ sophisticated methods of analysis exist 
to deal with such situations,^ and these are being explored. However, it was 
felt that -thh multiple regression approach, being more intuitively meaningful 
and interpret^le, would provide the best tool for initial explore^ ti on of the 
data. 
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TO account at least in part for the non-linearity in the relationship between 
pretest and posttest measures, and the potential interactive effect that this 
non-linearity might have introduced in many of the equations, the analyses 
were conducted, using subsets of students, Baseji on pretest performance, 
students were separated into three analysis groups, the highest-^achieving 
25%, th^ middle 50%, and the lowest 25^, This* procedure made it poss^-ble to 
examine possible interaction effects between independent variables and initial 
achievement, (A variable could, for example, have a positive relationship 
with the outcome for the lo\y group, no relationship for the middle group^ 
and a negative relationship in the high group.) This subgroup analysis also 
provided for the fact that certain types of insttuctional programs or materials 
might tiot have worked equally well or in a similar manner for different kinds 
of studeijts. To identify program variables that may have been important for 
all students, regardless of initial status, ^ analyses were also performed on 
the total sample of students.- 

C. RESULTS OF OVERALL REGRESSION ANALYSES 

The detailed results, of the analyses are presented in*" the Appendix, Tcxbles A-31 
through A-36. , These tables present, for eacfi evaluation sample .and outcome 
measure, ^ breakdown of the significant independent variables by grade level 
and smalysis subgroup (total, high-achieving, medivun-achieving, and low- 
achieving). The significance of a particular variable is indicated by the 
size of the Beta weight associated with that variable (shown in the corre- 
sponding cell of the matrix) . Values are shown in the tables for all analyses 
in which the F-ratio was greater than 3.90 in the final regression equation. 
Beta coefficients are reported for these analyses, because the dependent 
variable was in the form of a residual score. ^ 

IT' 

The discussion that follows summarizes patterns of significant relationships 
between independent varicibles and outcomes, where those patterns show some 
consistency across analysis groups and grade levels. Isolated cases of a 
significant relationship are not emphasized, since such results are too often 
spurious. The results are organized by the three major sample groupings (Pilot 
elementary seimple, Basic elementary sample, and Basic secondary sample). 

\ 

!• PILOT ELEMENTARY SAMPLE 
t 

The principal funding of the Pilot sample analyses was a pattern of associations 
.between supplemental funds, the nuipber of hours of mathematics instruction, and 
the reading and mathematics outcomes as measured by the achievement tests, A 
particularly noteworthy result was that increases in jsupplemental funds for 
reading were. associated with higher test scores in both mathematics and reading • 
Total time devoted to mathematics instruction vstas negatively related to reading 
outcomes, but showed no tclation to mathematics outcome^. The per-pupil regular 
furfds were positively associated with reading performance at the fourth-grade 
level, but negatively associated across grades with mathematics performance. 
Higher mathematics scores were associated with lower pupil/teacher ratios and 
with higher total school enrollments. 
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No strong patterns, of associations were found between^ the reading and 
mathematics outcomes and either the program or teacher characteristics used 
in these analyses. One student-level item, which indicated whether students 
felt they belonged in the school, showed a positive associajiion with, mathe- 
matics outcomes; that is, students who said they felt they belonged had 
higher derived mathematics scores. In addition, the Pilot sample showed 
associations between racial-ethnic group and reading scores; for' the average 
and better-than-average pretest groups in the fourth and fifth grades, non- 
minority students scored' higher than minority students. There was no * ' 
association between racial/ethnic group and reading outcomes for the students 
in the lowest quartile on pretest performance at these two grades, and no 
association for any students at the third grade. Moreover, no associations- 
between race/ethnicity and the residu^lized mathematics scores were found at 
any grade level in these -Pilot samples. 

2. BASIC ELEMENTARY sample' 

'One fairly clear and potentially" important jpattern was found in the analyses' 
of reading and mathematics outcomes for Basic elementary schools; this was in 
the relationship between achievement test scores and extent of the school 
activities associated with the desegregation process* For both the reading 
and the mathematics analyses, there was a positive relationship between the 
extent of these activities and the outcome variable for the third- and fourth- 
gr^ide levels. At the fifth grade, there was no association between these 
activities and reading outcomes, and there was a negative association with the 
math scores. ✓ 

A negative relationship was found between the extent to which comprehension 
skills were stressed in reading instruction, and reading scores for the average- 
level students. In mathematics, higher scores were associated with increased 
emphasis on diagnostic testing in the mathematics .programs. Contrary to the 
results from the Pilot sample, the Basic elementary data showed a weak pattern 
of negative associations between school enrollment and mathematics scores; that 
is, larger total enrollment appeared to be associated with lower scores. 

3. BASIC SECONDARY SAMPLE 

The strongest and most policy-relevant relationship found in the secondary- 
level samples was a pattern of positive associations between the amount of 
time spent in reading instruction and the reading achievement scores. This 
pattern was strongest for the students in the lowest quartile on the pretest 
performance, and ,was evident at all three grade levels, tenth through twelfth. 
The second clearest pattern in these secondary grades was a positive relation- 
ship between reading outcomes and school enrollments, particularly for students 
who were average or above in pretest scores. 

There was a moderately clear pattern" of positive associations between the 
student I s sense of control over his circumstances and his outcome scores . • 
in reading and mathematics. Students who felt their efforts would have a 
substantial impaction what happened to thenf tended to have higher residual 
achievement scores. This relationrship was strongese for the students with 
poor pretest performance. ^i' - 
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Only one pattern of associations was fojand between program characteristics 
and mathematics outcomes ♦ This showed a negative relationship between mathe- 
matics scores and the prc^ortion of mathematics instruction devoted to puzzle- 
and game-type activities. 



D. IN-DEPTH STUDY OF RELATIONSHIPS BETWEEN STUDENT OUTCOMES AND 
DESEGREGATION ACTIVITIES 

Section .0, above, examined actual or hypothesized relationships between a 
single type of outcome measure (£est performance) and a wide variety of 
program dimensions. One of the strongest findings, for the third and fourth 
grades of Basic elementary schools, was that desegregation-related school 
activities were positively associated with achievement test scores* This 
finding appeared of sufficient interest to warrant the use of further 
exploratory analyses, for the purpose of Examining possible explanatory 
mechanisms underlying the apparent effects of desegregation-related ^tivities 
on achievement* The present section of this chapter summarizes the methodo- 
logy and results of such additional analyses, all of which centered around 
measures related to the desegregation objectjLves of the ESAA Basic program 1 

Results reported here are based on analyses of Basic tfhird-grade data, because 
of the larger ^sample size at that grade level. (Only Basic programs were con- 
sidered because the desegregation objectives were not pertinent to the Pilot 
schools. Secondary-level data were excluded because of their poor quality; it 
was fovind that. many of the questions used to collect data on extent of classroom 
integration were inappropriate for high school classes.) Altho^gh preliminary 
analyses at the fourth and fifth grades indicated trends generally similar to 
those for the third grade, there were some differences in specific analyses. 
For this reason, the third-grade results reported here should not be considered^ 
totally generalizable to other grade levels. 

The analyses focused on the student-level impact of school activities concerned 
with desegregation, and* on related attitudes of school authorities. They used 
three indicators of the student's attitude toward education and his school as t 
criteri^on variables.* The first indicator was the response to an item asking 
if the student felt that he or ^he belonged to that school. The second variable 
of ' the criterion set was a six-item composite indicating the extent to which 
the student expressed a liking of his school. The third indicator was a 
variable reflecting 1:he student's expressed expectation of how much additional 
"education he woul^ obtain. 

The basic^ rationale for these analyses was that positive feelings indicated 
by these criterion vap.ables were desirable outcomes in their own right, and 
should be related to other desirable student outcomes, including achievement. 
The exploratory procedures used a three-step statistical analysis to* relate 
the criterion variables to four other set^ o^ indicators. The first step 
attempted to relate school administrative practices and actions to desegrega- 
tion-related teacher attitudes. Specifically, it investigated the linear 
relationship betweeh (1) »a- set of three indicators of teachers' attitudes 

*A brief description of the construction each variable or indicator referred 
to in this discussion is given in the Appendix, Table A-37. 
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toward desegregation issues and (2) another set of indicators of the actions 
and attrtudes of school administrators. 



The second step was concerned with interactions between district/school 
variables and student interracial variables. It examined relationships 
between (IT' the teacher-attitude indicators, plus indicators of the districts* 
desegregation-related activities and of principals* and superintendents*' 
attitudes, and (2) three measures of the extent of interracial activity among . 
the students. ^ 

The third step of the analysis dealt with possible effects of district, school 
and student-level desegregation-related variables on student attitudes toward 
school. It investigated the relationships between (1) the criterion measures 
of student attitudes toward school and (2) the combined indicators of the 
other four sets of variables (i.e., extent of student interracial activity, 
teacher attitudes, principal and superintendent attitudes, and district and 
school desegregation-related activities). The relationship of the students* 
attitudes to their mathematics and reading test scores was not directly 
examined in this set of analyses, although it seems logical that improved 
attitudes might have led to improved achievement; subsequent analyses based 
on longitudinal data will specifically explore this premise. 



The statistical procedure used for the analysis reported here was canonical 
correlation. The canonical corrqlational model is a procedure for examining 
the relationship between two sets of variables with each set having two or 
more variables. Given two such sets of variables, e.g.. Set A and Set B, 
the canonical, correlation yields the linear combination of variables in Set A 
that*produces the highest correlation with any possible linear combination of 
variables in Set B. The linear combinations of the variables in each set ate 
expressed in a form like that of the standardized regression modely^ The 
correlations of these linear combinations with statistics reflecting the 
probabilities of such correlations, are given as outputs of the procedure. 
For example, in the first step of the analysis described below, one of the 
sets of variables included the desegregation-related actions of the school 
and district, the index of the superintendent's attitude toward desegregation, 
and the i;idex of the principal's attitude toward desegregation. * The other 
set of variables included the three indices of the teacher's attitude. The 
canonical correlational analysis yielded the coefficients of the linear 
combination of school actions, superintendent attitude, and principal attitude 
measures that had the highest correlation ^with any linear combination of the 
teacher attitude measures. 

Several factors recommended the use of .canonical correlations for these 
analyses. A first consideration involved the nature of the items and" indices 
used. The analyses are primarily concerned with relationships among concepts 
that are operationally defined by the five sets of measures used. No single 
variable seemed to be adequate as a specification of the concepts involved, 
hence a model relating sets of variables was appropriate. The canoracal 
correlation mod^v^s developed for this ]^ind of analysis. Models involving 



1. 



METHODOLOGY 
O 




i43 



PX-9 



structural 'equations did not appear warranted, given the absence of any strong 
a priorj. conceptual models for these data. ^ 



»Since most of the items used in these analyses are defined at district, school, 
or teacher level , the analyses .used the school as the unit of analysis . 

• 2* RESULTS 

The first step in the canonical analysis focused on relationships between 
(1) attitude^ of the teaching staffs toward ESAA programs (i,e; , ^desegrega- 
tion-related programs), and (2) the position of the school officials as 
indicated by district actions and by the self-reported attitudes of adminis- 
trative personnel. While no direct causal relatedness was ^necessarily assxamed, 
some consistericy of the teachers' attitudes with the actions and attitudes of 
administrators was ^^ected. Such consistency might be the product of many 
factors, including, possib]ry, a causal impact of ^higher-level actions ^nd • 
attitudes on teachers, 'Conversely, the absence of such consistency might' 
indicate a serious conflict in bqiiefs within the school. 

Table IX-1 shows the result of th^ canonical correlational analysis for. three 
sets of variables using the aggregated school-level data from the third-grade 
Basic sample. This is interpreted as showing those linear- cpmposites of the 
two groups of variables that yielded the highest correlation. Thus, a 
composite school action/attitude index w^ obtained for each school by summing 
the products of the values of each 'variable and their apprdpriate factors 
(e,g,, the value of the index of desegregation multiplied by .11, plus the 
percentage of minority teachers multiplied by ,42, etc.). In a similar way, 
a composite teacher-attitude index was calculated for each school , The 
resulting composites were correlated with a. r = 0,48, 





V 






TABLE IX-1. CANONICAL CORRELATIONS: SCHOOL 
ACTIONS WITH TEACHER ATTITUDES 


/ 


Canonical 
Correlation 


Chi-Square 


Degrees of 
Freedom 


- ,48 


35.26* 


12' 


School Actions 




District Desegregation-Related Activities 
District Percentage Minority Teachers , 
Superintendent Attitudes 
Principal Attitudes 


.11 ' 
.42 
.90 
.24 


Teacher Attitudes 




Teacher Attitudes: Compensatory Education 
Teacher Attitudes: Cultural Enrichment 
Teacher Attitudes: Integration 


.54 
.00 
.61 
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These results show, the general consistency anticipated between school actions 
and teacher attitudes* The relative magnitude of the coefficient within a 
given composite ^is an approximate indication of the importance of' the variable 
in tile composite. Thus the composite made up of the school actions and the 
prihcipal and superintendent, /Attitudes was dominated by the superintendent- 
attitude measures. The percisntage of minority teachers in the district 
and the principal-attitude /measures were of somewhat lesser importance. 
The variable indicating the extent of the district activities to smooth 
the desegregation process contributed fairly little to the composite. 

High scores on the composite of teacher attitudes primarily reflected scores 
on the integration-desegregation index and a belief in the efficacy of 
the compensatory education programs. The teacher's belief in the. 
effectiveness of cultural enrichment programs showed no .contribution . 

Overall, these results indicate that the major element in the consistency 
between the two sets of variables was the similarity of the attitude toward 
desegregation on the part of the teachers and the school superintendents. 
, Similar analyses for the other elementary grade levels in the Basic schools 
tended to confix this result. 

The second step in the analyses was centered on the relationship between 
(1) the combined school staff's attitudes and actions, and (2) measures of 
the extent of integration of the pupils within fhe school settings. As in 
the pre\/ipus step, a/ consistency was anticipated. More extensive and more 
successful integration of' pupils would be expected in those schools where 
the staff had positive attitudes toward these outcomes. 

The results of the canonical correlational analyses of these relationships 
are shown in^ Table IX-2. One set of variables was the combination of atl 
seven measures exaanined in the previous analysis (i.e., the two mefisures 
of school actions, the three teacher-a^t'titude measures, and the indices of 
the superintendent's and the principal's attitude toward desegregation). The 
other set included three indicators of the extent of inter-ethnic relation- 
ships among the students themselves. Two of theste inter-ethnic measures 
reflected the teacher's assessment of the extent of the intergroup activity, 
of the sample students. One measure indicated the extent of the inter- 
action of the student with students of other ethnic groups in class 
discussions and projects. This measure was an indicator of the amount of 
such interactions in recreational activities at the school. The third » 
measure ih this set was an index of the' extent of friendly interactions 
between Black and White students in the school as a Whole. (The measure 
of the extent of interactions between Spanish-descent and Anglo students 
could not be used because it was inapplicable to more than* half the sample 
schools* ) 

Both canonical variates in Table IX-2 showed some degree of bipolarity. For 
the composite of the measures of within-school student integration, a high 
score seemed to reflect increased inter-ethnic interaction , in the classroom 
setting, and a lower score indicated more interactions in plby activities. 
The moderate-to-high positive coefficient for the measure of Black-White 
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interactions would seem to indicate that these interactions most often 
occurred in the formal classroom settings.* 



TABLE IX-2. CANONICAL CORREIATIONS OF SCHOOL ACTION 
AND STAFF ATTITUDES WITH CLASSROOM INTEGRATION, 



Canonical 

Corre] ation ^ Chi-Square 
0.77 ' 103.23 


Degrees of. 
Freedom 

21 


School Actions and Staff Attitudes 


District Desegregation- Related^ Activities 
District Percentage Minority Teachers 
Superintendent Attitudes. 
Principal Attitudes 

Teacher' A'lititudes: Conpensatory Education 
vTeacher Attitudes: Cultural Enrichment 
Teacher ^ttaftudes: Integration 


-O.Ol 
-0.07 
0.74 

r-0.24 

0.36 
0.O2 
0.08 


Class rooDii^Integration. 


« ^ > 

Student Interracial Interactions: Recreation 
Student Interracial Interactions: Classroom 
Black and White Student Interactions 


0.17 
0.72 
0.56 



The meaning of the coiiposite of school actions ^nd staff attitudes is best 
described by the high scores of this variate. High scores reflected pro- 
integration/desegregation attitudes on the part of the superintendent and 
a belief in the effectiveness of , the compensatory programs by the teachers. 
The low end of this ^f^posite measure was less clearly dfefined, but it 
appeared to be associated with principal attitudes that were favorable to 
desegregation. ' 

Considered together, the two canonical composites suggest that schools with 
relatively more intergroup activity within the formal instructional sessions 

*The presence of this bipolar scale should not be interpreted to imply that 
more classroom intergroup activity means less recreational intergroup behavior 
and vice versa. The high positive correlation between these two* measures 
(r = 0.85) shows this is not the case. Rather, this should be interpreted 
to mean that the part of these measures of intergroup activities that is 
accounted for by the composite is related in a bipolar manner to the composite 
of the other variables. ^ 
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tended to have a superintendent and teachers with more favorable 'attitudes 
toward integration/desegregation. In schools where the principal had 
relatively favorable attitudes toward desegregation, thei:e tended to be 
more informal interaction among students of differing ethnic groups. It 
must be emphasized/ however/ that these canonical analyses do not necessarily 
show direct causal relationships. They simply demonstrate one possible set 
of linear associations/ and do not specify the structure or direction of any 
' interactions. 

The third gtep of the canonical analysis focused on the degree of association 
between (1) a set of variables combining all the measures used in the prior 
two analyses, and (2) a set of three measures of the student's satisfaction 
With, and his expectations of, his school experiences. The first set was 
composed of two measures of desegregation-related school actions, measures 
of staff attitudes related* to ESAA-type programs, and three indices of the 
extent of integration of students of- different groups in the school 
activities. The composites of these variables reflected some of the major 
features of the school environment which fqrmed the educational context for 
the student. - - ' ^ 

The second set of variables consisted of an index of the extent of additional 
schooling the student expected, a measure of his feeling of belonging in^ the 
school/ and an index of student's liking of his school experiences. Positive 
values for these variables presumably reflected desirable student outcomes, 
and these outcomes^, in turn/ might logically be expected to influence and 
interact with other desired student outcomes, including achievement. ; 

Table IX~3 shows that the two composite measures were highly correlated 
(r = 0.90) and that the relationship was strongly positive; that is, the major 
coefficients were positive. The composite of the context measures was pri- 
marily a function of the inde:c of the teacher's belie'f in the vnlup of 
compensatory programs/ the teacher's score on the integration-desegregation 
index/ and the extent of the integration of the students in formal class- 
room activities. The composite of the student variables was almost entirely 
a function of the measurd of the student's educational expectations / with the 
student's liking for school contributing a very small positive component. 

The results suggest that a student's educational expectations may be^strongjly . 
influenced by the nature of the environment as determined by desegregation- 
related educational experiences. They further indicate that the teacher's ' 
attitudes may be a particularly important component of this influence.' This ^ 
interpretation was reinforced by similar findings for the other elementary 
grade levels in the Basic samples. * 

3. SUMMARY , I 

The canonical correlation analyses discussed here did not directly seek to , 
relate specific program characteristics to enhanced student achievement test 
performance; had achievement been included in these analyses, they would have 
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TABLE IX-3. CANONICAL CORRELATIONS OP SCHOOL ACTION, STAFF ATTITUDES, 
AND CLASSROOM INTEGRATION WITH STUDENT ATTITUDE TOWARD SCHOOL 



Canonical 
C9r relation 

^ 0.90 



Chi-Square 
144.92 



Degrees of 
Freedom 

30 - 



School Actions, Staff Attitudes/ and Classroom Integration 



District Desegregation-Rel^fred^ctivities 
District Percentage Mir^j^ioty Teachers 
Superintendent Attijfeti^s 
Principal Attitpd^ 

Teacher Att^^fetides: Compensatory Education' 
Teacher^t£itudes: Cultural Enrichment 
Teachier Attitudes: Integration 
SJ:uaent Interracial Interactions: Recreation 
'Student Interracial Interactions: Classroom 
Black and White Student Interactions 



0.08 

0.07 

0,18' 

O..63 

0.60 



-0, 
0, 

-0, 
6, 

-0, 



07 
41 
21 
32 
11 



Student Attitudes Toward School 


Students' 


Feeling of Belonging 


0.00 


Student's 


Liking for School / 


0.11 


Student's 


Educational Expecytations 


0.92 



been largely redundant to thef overall exploratory analyses (i.e., regression 
analyses) reported in Section jC of this chapter. Rather, the canonical 
analyses investigated relationships among various levels of district, 
school, teacher, and student activities ind attitudes presumed to be related 
to desegregation and intego/ation. The gbal of these analyses was to seek 
meaningful patterns of inter-variable relationships that might suggest 
possible causal linkages wa thirl a sequence ultimately connecting program 
activities wii;h student achievement. 

In summarizing the results of the canonical correlation analyses, it should 
be noted that there were statistically significant and relatively consistent 
patterns of associations among sets of variables in each step of the analysis 
reported above. 

Although there was no direct evidence of causality, because of the non- 
experimental nature of the analytic design, the results are at least suggestive 
that (a) positive superinytendent attitudes towar^ de'segregation were associated 
with positive teacher atyitudes toward integration and compensatory education; 
(b) positive superintendent and teacher attitudes toward integration and 
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compensatory education were associated with more frequent student interracial 
interactions, particularly in'classroom situations? and (c) positive 
superintendent and teacher attitudes and frequent student interracial inter- 
actions Vere associated with high educational expectations on ,the part of 
students and with students' liking for school. 

Should these relationships be confirmed in subsequent analyses of the second- 
year and third-year data, and especially should the positive stude^nt attitudes 
be found related to higher test achievement, these findings will have clear 
implications for the He«\gn -and implementation of future ESAA-like programs. 
Specifically, they will strongly suggest that positive 'attitudes and activities 
of district/school administrators and teachers, with respect to desegregation- 
related issues, are important factors ip promoting positive student attitudes 
toward school, and that these attitudes in turn favorably^ affect the students* 
cognitive, achievement. Even if the final linkage between student attitudes 
and achievement should fail to be directly substantiated, it seems clear that 
more positive interactions among students of different racial/ethnic groups,' 
and more positive feelings toward school, are important outcomes in their 
own right. In either event, there would appear to be suggestive evidence 
that ESAA-like programs should place fairly strong emphasis on activities 
designed to develop more positive attitudes toward desegregation within 
administrators and teachers. 
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CHAPTER X 



SUMMARY OF RESULTS 



Before reviewing the major findings of the report, two important facts should 
be noted. First, this report deals only with the first year of ESAA^ imple- 
mentation (school year 1973-1974) . Siibsequent, reports will describe program 
impact two and three years after initial implraientation and cumulative impact • 
over three school years. Second^ because of difficulties in obtaining outcome 
data that mijght more directly reflect desegregation activities in the first 
year of the study, the present report is based on a single set of criterion 
measures; achievement test scores in reading and mathematics. Criterion 
data collected for the second and third years* evaluations will include 
measures ^of racial climate and of reduction in minority-group isolation, in 
addition to the achievement scores. 

The remainder of this chapter first summarizes some of the more relevant * 
characteristics of the sample districts and schools, including the racial/ 
ethnic compositions of their staffs and student bodies, and the desegregation 
activities of districts having Basic programs. Descriptive data on the back- 
grounds and needs of sample students are then presented, followed by infor- 
mation on funding allocations and othei: program characteristics in the sample 
districts and sqhools. Finally, results of analyses of the sample students* 
gains in achievement test ,scores are summarized and interpreted. 

A. CHARACTERISTIC S OF SAMPLE DISTRICTS AND SCHOOLS 

The first-year evaluation samples included 76 Basic elementary schools 
(treatment plus control), 30 Basic secondary- schools, and 62 Pilot elemen- 
tary schools in 80 ESAA-funded school districts. For the combined samples, 
over half the districts were in the Southeast and South Central regions of 
the country; there were no sample sites in New England or in the Western 
Mountain region (Colorado, Montana, etc.). 

Each sample was drawn from an evaluation universe that was defined to maxi- 
mize the study's ability to meet major evaluation objectives. (See Chapter I 
for a summary of study objectives, and Chapter II for details of the site- 
selection procedures.) Sampling procedures were designed to ensure that all 
scunples were fully representative of their respective evaluation universes. 
Relatively little information is av.ailable concerning the extent to which the 
samples were representative of the total universes of Pilot and Basic ESAA 
awards. With respect to regional distribution, the sample districts were 
highly representative of the ESAA award universe. That is, the number of sample 
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Iites in different regions was sttongly correlated with the number of grant 
awards to those regions. However, because of the emphasis on obtaining matched 
pairs bf schools and adequate samples of students within each district, small 
districts tended to be eliminated; as a result, the sample districts werg,^n 
the average, larger than the Universe of ESAA districts. In addition, ^mjple 
Basic districts tended, to have poorer matching of racial/ethnic proportions at 
'the district, school^ and class levels than the universe of districts receiving 
Basic grant awards. , 

The racdal/ethnic composition of sample Pilot schools (by definition minority- 
isolated) showed predonp.nant^ minority-student enrollment, while sample Basic 
schools in general werq about evenly divided in overall majority /minority-group 
rcprcccntation among students. In both Basic and rilct schools, the percentages 
of minority-group teachers and administrators were consistently lower than the 
percentages of minority-group students. There was a better correspondence ^• 
between staff racial/ethnic composition and student boidy, racial/ethnic compo- 
sition irt sample Pilot proqrams than in the Basic programs. 

There was a considerably higher average pejfcientage of economically disadvantaged 
students (defined in terms of eligibility for participation in Title I programs) 
in sample Pilot schools (59%) than in sample Basic elementary (39%) and secondary 
(30%) schools. Community involvement, as indicated by the percentage of paid 
paraprofessional and clerical staff drawn from community resources, was greater 
in Pilot and Basic elementary scjicjbls than in Basic secondary' schools. 

Decision-making within the sample districts was relatively decentralized for 
decisions concerning classroom curricula and media; parents and students, as 
well as teachers and administrators, reportedly participated in such decisions 
in a. fifth of the elementary schools and half the secondary scjiools. Decisions 
concerning budget expenditure's were typically shared by a considerably more 
exclusive group, with very little parent or student involvement. Finally, 
personnel decisions were usually limited to participation by district and 
school administrators. 

Elements of district desegregation plans most, frequently reported by superin- 
tendents in the Basic sample districts were bussing of students from other areas 
and drawing of new school boundaries to provide for desegregation. Assignment 
of pupils by geographic attendance area was more likely to occur in^^districts 
characterized by moderate or higji percentages of minority students. Bussing 
of students to achieve greater racial balance was more likely to occur in 
'di-stricts that had small percentages of minority students or' small total- 
enrollment sizes. There was also some evidence that bussing was associated 
with districts that had grec^ter racial balance (higher Desegregation Index 
values). In those districts that used bussing, it appears that larger per- 
centages of minority students than non-minority students v;ere bussed. 

Overall, the amount of desegregation that took place during ESAA Year I was 
small; this was not unexpected, 'since most of the Basic award districts had 
been desegregating their school systems for two or more years prior to the 
creation o^ the ESAA program. However, 20% of the sample districts did achieve 
relatively large reductions in minority-group isolation during the evaluation 
year. 
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Treatment-control comparisons within the Pilot and Basic sample groups showed 
virtually no differences in any sociodemographic dimension, indicating con- 
siderable success of the-'-^^luation in randomizing treatment and control 
school assignments to meet the experimental condition of the sampling design. 

S TUDENT BACKGROUNDS AND IjEEDS 

Sample students were almost evenly divided betv;een males and females. Within 
the Pilot sample, approximately 68% of the students were categorized as Black 
by their teacher,s, 15% were of Spanish background, and about 13% were White. 
In the Basic elementary sample, roughly 44% of the students were Black, 8% 
of Spanish background, and 45% White. Of the Basic secondary students, 45% 
were Slack, 54% were^hite, and only .6% were o£ Spanish background. The 
small percentage of^ Spanish-background students at the secondary level may, 
in part, reflegfe a higher dropout rate among that group ii\ high school;, in 
addition, the sajjiple di<k n9t' include 'any large districts in areas of the 
country with sizable prbportions of Spanish-background students. 

The academic needs of the sample students were clearly evidenced by their 
pretest performance on the achievement tests in reading and mathematics. 
^Though performance' varied somewhat across»jgrade levels and across evaluation 
samples, all of the mean scores were well below the 50th percentile, with 
the bulk lying between the 16th and 37t^ percentilest^Beaddrig vocabulary 
and mathematics concepts were particularly weak areas for the sample students 
in both Basic and Pilot programs. These results appear tcx-^rn,dicate that the 
ESAA grant award process led to selection of needy districts aad schools, and 
that the schools selected needy students to participate in the local ESM 
programs. It was impossible to determine, however, whether t]ae selected 
schools and students were the most needy in the participating districts. 

Most sample s,tudents were also socioeconomically disadvantaged. Fewer than 
half of the students' parents were repotted to ^have completed high school, 
and a large percentage of parents had unskilled jobsl On the average, <• 
families of minority students were n\ore disadvantaged than those of non- 
minority students. " 

Puse of the random selection procedures xiseQ to select students, the 
les of students can be assumed representative of ESAA-eligible students 
he selected grade levels in the sample *schQols^^TP;t:eatment and control 
ps were j^uite effectively matched, as 'eviden&ea^^y the almost total 
a;:)sence of significant differei^s between those groups on demographic 
variables or on pre test /scores . 
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ROG Ri\M CHAPiVCTERISl 

Particular emphasis was placed on funding- allocations of the school prpgrams 
because of tlie importance of expenditure data in the design of the evaluation. 
Information wa's collected on all expenditures in both treatment and control 
schools, since tinalyses of ESAA expenditures alone would be meaningless without 
knowledge of the total bu'<^getary context. 

S^np^^districts had average total per-pupil expenditures of $1,343, and 
average 'Supplemental per-pupil allocations of $460. At the school level. 
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the average per-pupil total expenditures were $1,031 for the Pilot sample, 
$994 for Basic elementary schools, and $879 for Basic secondai^ schools'* 
The total supplemental funding levels were $238 per pupil for the Pilot 
sample, $120 for the Basic elementary sample, and $32 .for the Basic secondary 
sample. 

ESAA funding, a key variable in this study, ranged from a high ♦of $231 per 
pupil for the Pilot treatment schools to a low of $60 ^per pupil in the Basic 
secondary treatment schools. The pattern of allocation of ESAA funds within 
schools was somewhat different than that of non-ESAA supplemental funds. At 
the elementary level, a larger portion of ESAA money than of other supple- 
mental money was used for a cluster of activities that included counseling 
and guidance^, community activities, intercultural relations programs, etc. 
In contrast, non-ESAA supplementary funds were focused more heavily than 
ESAA funds on reading activities. At the secondary level, ESAA funds were 
concentrated niore h§avily (relative to non-ESAA funds) on reading, cind non- 
ESAA funds emphasized mathematics more strongly. 

Local Pilot programs were supposed to focus entirely on improvement of basic 
skills, and approximately two-thirds of the Pilot funds overall were repor- 
tedly spent on activities (Jirectly addressing that objective. The remaining 
third of the funds were committed to a vari^y of supportive activities such 
as individual and group counseling and guidcince of students, nev/ curriculum \ 
development, cc^mmunity activities, and administrative costs,. More detailefU \^ 
study is needed to determine the degree of relevance of those supportive 
activities to improvement of basic skills. In Basic elementary and secondary 
schools the allocation of funds across the three major applicatioa areas (read- 
ing, mathematics, and "other" activities including desegregation-related activi- 
ties) appears reasonable in light of the Basic program's Congressional mandate to 
pursue desegregation-related goals as well as basic skills improvemdn^jp 

A fundamental premise of the evaluation's experimental design was that a* 

treatment school's ESAA funding would constitute an increment over and above 

the funding level of a matched CQntrdi school. The validity of this premise . 

was examined by the use of paired t-tt2Sts on^he tot^l per-pupil supplemental 

funding levels. (It was assumed that regular funding was constant across 

schools within a district, and thus the only variability in total funding 

would be in. the supplemehtal. ftinding ) — 



A statistically significant funding difference in favor of the treatment 
schools yas found f6r the Basic elementary and Basic secondary samples, but 
not for the Pilot sample. Only about a third of the Basic treatment schools 
had a per-pupil funding level exceeding the level in t*he, matched control 
schools by more than 5%. Though the educational interpretation of this level 
of funding increment is highly judgmental, it is certainly not clear that a 
5% dif feifentic^l should be expected* to bring dramatic changes in student 
achievement. 

^- .1 • 

Patterns of -suppl^ental fund allocation were compared for the treatment and 
control groups. At the elementary-school level, the percentages of funds allo- 
cated to the reading, mathematics, and\ "other" categories (e^g. , counseling, 
intercultural activities) were similar for the treatment and control groups. 
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For^both groups, funds were focused most heavily in the reading area. At 
the secondary level, the trearinent schools placed heavier concentration of 
funds in reading, while the control schools placed more funds in the "other" 
area. 

Presumably ^e effectiveness of ESAA funds or of any other school funds 
depends on rov those funds a^ire applied to resources, and on the goals and 
operating characteristics of the school programs. Data on these program 
characteristics were collected for both treatment and control schools, and 
for both regular and remedial instruction. 

Within the sample elementary schools, inservice training was mos,t heavily 
focused on reading, with about 50% of the sample teachers participating. 
Training in the instruction of the disadvantaged had the lowest rate of 
teacher participation (about 20%). Basic secondary schools reported less 
inserv.ice training ov^all than elementary schools, but placed proportionately 
greater emphasis on training associated with intergroup relations. 

It r 

At the elementary school level, most teachers had the bachelor's degree, and 
from 14% to 26% (depending on the subsample) had master's degrees. In sample 
secondary schools, almost half of the teachers had advanced degrees. The 
percentages of minority staff members were consistently lower than the corre- 
sponding-percentages of minority students. 

In sample elementary schools, remedial reading teachers were more frequently 
available than any other type of specialized support staff, while at the 
secondary level, the relatively small number of remedial teachers was evenly 
divided between the reading and math areas. Aides were more frequently availr 
able., in Pilot programs than in Basic programSii 

By far, the majority ^of treatment-control comparisons showed no significant 
differences in availability of resources. The more important exceptions to 
this rule were that (a) in the Basic elementary sample, treatment schools had 
greater availability of ins.ervice training ^and of teacher aides for reading 
and math instruction, and (b) in* the Basic secondary sample, more treatment 
group teachers* than control group teachers participated in intercultural 
training sessipns. 



More detailed data were collected on specific characteristics of the reading 
and math programs, with particular emphasis^on the amount of student exposure 
to different types Of instruction. Sample Pilot students received slightly 
greater average total weekly exposure in reading instruction (11.4 hours) 
than sample Basic elementary students (10.3 hours). ^Exposure to math instruc- 
tion was similar for, the PilQt (8.1 hours) and Basic elementary samples (8.0 
hours). At the secondary ^level, total weekly exposure to reading instruction 
was 8.1 hours, and exposure to math activities was 5.7 hours. 

Regular reading progrcims emphasized objectives related to improving compre- 
hensipri^ skills and basic word techniques; remedial reading programs stressed 
those same skilds but aT&tJ^f ocused heavily on development of sensory-perceptual 
motor skills. Both regular and remedial math programs emphasized objectives in 
math concepts and ope^rations, and gave .considerable ^priority to efforts to > 
increase student motivation to learn math. 
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The size of^most regular elementary-level instructional classes in reading 
and math ranged from 21 to 25 students; at the secondary level, the classes 
were slightly larger. Remedial classes were somewhat smaller (16 to 20 
.students) , and there was greater emphasis on individualized instruction, 

Treatm^ht-control comparisons were made on a large number of variables related 
to the reading and math instruction, and in most instances no significant 
differences were found. One exception, however, was that ij) sample Pilot 
programs, treatment stuc^ents spent a larger percentage of their time than 
control students being tutored by an older person. Also, ^n sample Basic 
elementary programs, instructional class size was significantly smaller in 
treatment schools than in control schools. 

Many of the sample schools reported scxne form of interdultural and counseling 
activities. In both Pilot and Basic elementary sample schools, the heaviest 
concentration was on cultural enrichment programs , followed by field trips and 
group counseling activities. At the secondary level, there was heavy emphasis 
on field trips and individual and group counseling,, followed by cultural en- 
richment and interracial programs . / 

Within sample elementary schools receiving ESAA funds, a consistent relation- 
ship was found between students* pretest scores on the achievement subtests 
, and amount of exposure given to the students in different areas of academic 
content. This relationship indicated that, at least at a gross level, the 
ESAA programs were appropriately focusing their instructional efforts in the 
areas of greatest student needs. , 



D. STUDENT GAINS IN ACHIEVEMENT TEST SQORES 

Two sets of analyses were performed to assess the overall gains of students 
in the evaluation samples. First, test scores of the paired treatment (ESAA- 
funded) and control (non-ESAA) schools were compared, in an attempt to identify 
possible Differential effects of the ESAA funding. Second, the test scores of 
the total evaluation samples (combined treatment and control schools) were 
examined to determihe whether the overall pretect-posttedt gains were greater 
than might have been predicted on the basis of prior nortetive data. 

Wi^h Y-<^gpc>c;t to the tyeatment-control congarisons, n o evidence was found of 
overall differences in achievement test scores. Such an outcome was rroi: 
surprising, in view of the brief, span oS program-.operation; the average pre 
test-posttest interval was only slightly more than five moriths, and many of the 
programs had just barely become operational at pretest time. Furthermore — 
and perhaps in part reflecting the early formative stage of the ESAA programs — 
there were few differences in the treatment and control programs themselves 
that might have led to an expectation of significant outcome differences. 
Treatmerrt-contro](. differences in total per-pupil funding were not significant 
in the Pilots sample; while statistically significant in the Basic samples, 
the actual amount* of the funding difference exceeded 5% of the control-school 
allocation's in only about a third of' the sample districts. Finally, although 
signiticant treatment-control differences were found in a few dimensions of 
the schools* operational programs, these were few in number considering^ the 
many different combinations of variables on which comparisons were made. In 
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short/ there was little evidence, overall, that the treatment schools weVe 
•markedly different than their paired control schools. 

In the analyse^ of achievement test gains in ,the combined treatment and 
control schools, the results were encou?; aging, but difficult to interpret. 
^'Expected*' gains, were defined according to three different criteria. 
One criterion represented the gain necessary for a student who spored 
at the median (50th percentile) on the pretest to maintain that » same 
relative position on the posttest;- determination of this cri^terion gain was 
based on the test publisher's national norms for a sample of the general 
population of students at the grade levels of interest. The second criterion, 
also based on the publisher's national normative data, represented the gain 
necessary for each student in the ESAA evaluation sample to maintain his same 
\relative position (whatever that was) from pretest to posttest. The third , 
criterion attempted to take into account the tendency of disadvantaged 
students to gain in achievement at slower rates than the general population 
as a whole. To define this third criterion, empirical data from the earlier 
restandardization testing were used tp establish overall growth rates for 
disadvantaged students (as defined' by the 4;e^anda^i:di2ation Scimple of ESAA- 
eligible students) ; those rate values were then used- as scale factors in a 
downward adjustment of the gains required to meet criterion (i.e., to 
maintain a median position) . 

These three criteria represent somewhat diffeifent levels of difficulty, with 
the first criterion (maintaining the 50th percentile) being generally the 
most stringent because it is based on the overall populatipn of students 
rather than on a subseunple of disadvantaged students. The use of the three 
criteria is extremely speculative, as it involves unsubstantiated assumptions 
concerning . the shape of the growth curves within and across school years. 
Thus, any results from analyses using the criteria must be interpreted with 
extreme caution, and should be regarded only as suggestive of genereil trends. 
Given these constraints, the analyses (which used Bayesian techniques to allow 
individual program comparisons) appear to indicate' that the sample students 
(treatment plus control) overall made gains at least as large as might have 
been "expected*' --on the basis of. the two "disadvantaged" criteria (second and 
third criteria) , and, for some conditions approached the more demanding 
50th-percentile criterion. The niunbers of schools reaching or exceeding 
criterion varied with the achievement subtest involved as well as with the 
specific gain* criterion used. At the elementary level overall, much larger 
gcuLns— were^mada- (X.~a>>^^argj^r_nmnbers^ of _schools met the criteria) ^n math 
than in reading. Basic schools tended to make larger gains than Pilot schools, 
but the'*differences were not dramatic. In the reading area, greater gains 
were made irf^ comprehension than in vocabulary. ' 

Additional analyses were performed examining gains aggregated across all 
schools rather than by matched pairs of schools; these analyses did not 
use Bayesian techniques but simply compared the 'observed and "expected" 
gains. Results v;ere quite consistent with those of the Bayesian analyses, 
and ag'ain indicated that gains were generally larger than anticipated for 
disadvantaged students. 
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How^should the results of the analyseff:of overall gains (in treatment plus 
control schools) be interpreted? One possible inference is that the ESAA 
funds, ^dded to the general pool of dollars available to the sampl^e districts, 
allowed the districts to increase the* total supplemental budgets^bf control 
schools as well as treatment schools (e,g'. , hy cyLving more of the Title I 
funds to the control schools) , and thereby helped to enha*nce achievement 
in both treatment and control schools* . However / another tenable inference 
is that the results of t^e Bayesiai), analyses are artifacts of saxi\pling 
or measurement errors, or simply reflect the use of unwarranted assumptions 
about the shapes of "expected" learning curves, The first-year data simply 
do not provide a suitable basis for moving beyond the conjectural stacfe 
with respect to the overall gains of the evaluation sample. It is hoped 
that later analyses of the second and third years' data will help at least 
to eliminate some of the alternative explanation^, assuming that the same 
ge^^eral pattern of results is founds 

E. RELATIONSHIPS AMONG ACHIEVEMENT SCORES, STUDENT VARIABLES, 
AND PROGRAM VARIABLES 

Linear regression analyses were performed to explore relationships among 
student characteristics, staff and program chai;acteristics , and achievement 
test scores. Before summarizing the more interesting results of these 
analyses, it is important to stress certain limitations in the analytic 
design that affect the interpretation of the findings. First, the regression 
analyses were not based on a true experimental design. Neither schools nor 
students were randomly assigned to the different treatment conditions being 
compared. As a consequence, it is quite possible that observed relationships 
may be artifacts of the samples involved in these analyses. For example, if 
"program trait "X" were found associated with higher reading scores, this 
might simply reflect 'the fact that, coincidentally , schools with trait "X" 
also had students who received more home support for their reading studies. 

The second constraint on interpretation of the regression analyses results 
is that nothing in these analyses can be interpreted as evidence of causal 
relationships between program variables and achievement test measures. The 
analyses yield estimates of the degree of association among variables,' but ,^ 
they do not provide any information about whether changes in one set of 
variables (e.g., instructional procedures) cause changes in another set 
(achievement test scores) . 

Finally, in no case was a relationship found statistically significant at 
all grade levels and for all test measures; however, the results summarized 
below involve relationships that were found across enough different grade 
levels and test criterion measures to be interpretable as showing some - 
moderately consistent tx^end. For summarization purposes the general trends 
are described below, but readers must refer to the more detailed discussions 
in the earlier chapters to ascertain the breadth and consistency of those 
trends. * 

Turning now to the results of the regression analyses, one finding of interest 
in the Pilot elementary sample was a positive relationship between level' of 
supplemental funding for reading instruction and achievement scores in both 
reading and mathematics. One possible interpretation of this finding is that 
a greater focus of supplemental funds on reading instruction had beneficial 
effects that generalized over both reading and mathematics achievement gains, 
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Such a .generalizatipn to mathematics would not be surprising, since mathematics 
instruction is heavily dependent on the students' mastery of langu'age skills. 

Should this causal interpretation be valid, there would seem to be clear policy 
implications for future ESAA and ESAA-like programs conducted in the context of 
minority-isolated elementary schools: where the improvement of academic test 
performance is an important program goal, supplemental funding should be heavily 
focused on the improvement of reading instruction. Us implied by the 
general caution given abov^e, however, there is no empirical data for 
inferring a causal relationship between funding level for reading instruc- 
tion and test pdrformance. 
j. 

A second finding, again in the Pilot elementary sample, was that lower 
pupil/ teacher, ratios and higher total school enrollments were associated 
with higher mathematics test scores . If one were to assume an underlying 
causal relationship, the fact that pupil/teacher ratio and size of school 
enrollment *'affedted" 'Oivly mathematics test scores could be explained on 
the theory that mathematic^scores v;ere potentially more sensitive to 
differences in program approaches than reading scores. Reading scores are 
known to be heavily influenced by students' experiences outside .the class- 
room, whereas mathematics scores have typically been found more responsive to^ 
variations in the formal classroom instruction. Again, however, there was 
no direct evidence of causality, and the relationship found here could 
easily be "explained" by^-a large number of alternative interpretations. 

In Basic elementary schools where, unlike the Pilot schools, a major program 
goal was the improvement of interracial relations, certain program character- 
istics related to desegregation were' found associated with achievement scores. 
At the third and fourth grades, schools that had a higher frequency of 
school activities , related to the desegregation process (e.'g*; frequency of 
public meetings, media presentations, and staff meetings concerned with inter- 
xacial_intexactions)„also tended to have higher achievement scores in both 
reading and mathematics. This finding is of speciai~interes€""Fecause~^^f^'t3 
parallel to certain findings in a previous study of the Emergency School 
Assistance Program (ESAP).* In that study, which dealt with a program quite 
similar to ESAA, certain indicators of positive racial atmosphere in the . 
schools were positively associated witlv<^achievement gains at both the fifth 
and tenth ^lades. A relationship was alfeo found in the ESAP evaluation between 
school integration (as measured by racial composition within schools) and 
achievement scores. Thus, both studies seem to suggest thc\t some form of 
relationship exists between variables associated with integration and racial 
attitudes, and student achievement test scores-- but in neither study was there 
any direct basis for interpreting the causal nature 6f that association. 

Another finding in the Basic elementary sample was a positive relationship 
in some of the subsainples between the frequency of diagnostic testing by the 
teacher and the students' achievement test gains in mathematics. This 
rela:tionship, though neither as strong nor as consistent as those described 
above, is of interest because, it was one of the few associations of even 
moderate strength involving specific instructional techniques. One may 

*The National Opinion Research Center. Southern Schools: An Evaluation of 
the Effects of the Emergency School Assistance Program and of School Desegre- 
gation . University of Chicago. . ^ 
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speculate that the more frequent use of diagnostic tests enabled teachers 
to pinpoint learning deficiencies ^of students and thereby to remedy these 
deficiencies more effectively. Should this fiMiog be substantiated in some 
future study ynder more experimentally-controlled conditions^ it would argue 
strongly that ESAA-like programs should establish guidelines and train 
teachers to. make frequent and systematic use of diagnostic tests. 

The strongest relationship found in Basic* secondary schools ,was a pattern of 
positive associations between the amount of time spent in reading instruction 
and the reading achievement scores. This pattern was strongest for students 
in thfe lowest quartile on pretest performance ,^nd was evident -j-n all three 
grade levels / tenth through twelfth* Given the usual caveats concerning 
attributions of causality in purely associative analyses / this finding seems 
to suggest that greater student exposure to reading activities may have led to 
enhanced reading performance* Should a causal relationship be demons trat'ed in 
future studies, this unsurprising finding would clearly support a recommendation 
that progrcims provide as much student exposure as possible to reading instruc- 
tion . 

f; relatiomships between staffs' desegregation-related attitudes and 
student attitudes 

As a result of the observed relationship between desegregation-related school 
activities and achievement scor^es in the Basic elementary sample, additional 
exploratory analyses were performed to examine the relationship between 
desegregation-related activities and attitudes at the classroom, school, 
and district levels, and student attitudes toward education and school. , ' 
Canonical correlation methods were used to examine the degree of association 

among, .sets ,o£- variables-^at_thr.e^e_qpn.cepixial sjtag.es^.in the _ tracing of the. . 

influence of desegregation-related attitudes and activities* At the first 
stage, favorable actions and attitudes of district administrators tov/ard desegre- 
gation were found associated with positive (pro-integration) teacher attitudes 
on desegregation-related issues. In the second stage of analysis, a greater 
degree -of positive interracial activity among students was f ounJ in districts and 
schools that had favorable teacher and superintendent attitudes toward integratipn 
and greater frequency of desegregation-related district activities. Finally, in 
the third stage, more positive attitudes toward school and toward their own 
educational futures were found in students in schools with greater amounts of 
student interracial activity, more positive teacher and superintendent 
attitudes on desegregation issues, and more frequent desegregation-related 
district and school activities* The overall pattern of these analyses 
suggests strong contextual effects of attitudes and activities related to 
desegregation in the classroom and school on the attitudes and expectations 
of students. Students ' educational expectations, in particular, may be 
strongly influenced by the nature of the educational environment, and teachers' 
attitudes may be a particularly important component of this environment. Al- 
though a linkage between positive student attitudes and enhanced academic 

•achievement might reasonably J^e-^hypothesized,. suck.a. linkage. was^_not_direcJtly 

tested in this series of analyses. 
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The discovery of strong relationships at each stage of these analyses does not 
provide evidence of causality; however, it does suggest important relationships 
between attitudes of administrators and teachers, between actions at the school 
and district level and student racial interactions , and between all of these 
variables and student attitudes. Should these relationships be confirmed m 
subsequent analyses of second- and third-year data, they will have important 
"implications for policy-makers attempting to effect changes in student atti- 
tudes. While positive student attitudes constitute an, important outcome m 
themselves, subsequent analyses will explore the influence of these attitudes 
on student achievement scores. ^ 
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TABLE AVERAGE WEEKLY PERCENTAGE OF TOTAL READING EXPOSURE 

SPENT IN INDIVIDUAL READING ACTIVITIES IN PILOT SAMPLE 

SCHOOLS 

* t 



m 

Activity 


Treatment 
(Total N = 14) 


Control 
(Total N = 14) 


Unpaired 
Treatment 
(Total N = 14) 


Total Weekly Reading 
Exposure (in hours) 


X 

11 . 56 


S;D. 
3 • 51 


X 

11.51 


S.D. 
3 .13 


X 

11.11 


S.D. 
1.99 


Component Activities 


%" 


S.D. 




S.D. 


? 


S.D. 


Group Instruction 


■ ■ r 
22-. 9 •■ 


7.1 


24.9 


9.3 


24 -^-x^. 2 


Tutoring by Older Person 


8.0* 


3.8 


5.3 


3.2 


. 7.6 


3.1 


Peer Tutoring 


3.4 


2.4 


3.5 


1.9 


2.8 


1.7 


Machine-^Mediated 
Instruction ^ 


5.4 


4.1 


5.8 


2.4 


B.6 


3.2 


Games/Contests 


5.8 


3.4 


6.4 


2.4 


6.4 


2.2 


Diagnostic (Test Taking) 


5.1 


1.7 


5.1 


2.2 


5.8 


J. 6 . 


Independent Seat Exercises 


18.6 


6.4 


19.0 


4.6 


18.8 


4.4 


Report/story Writing 
Individualized Reading 


6.5 


2,8 


6.4 


1.9 


5.2 


2.8 


15.7 


9.1 


14.2 


4.9 


13.2 


3.2 


Student Presentation 


3.5 


3.3 


4.3 


2.2 


3.1 


1.4 


"Pullout" Compensatory 
Reading 


5.3 


4.2 


6.2 


5.9 


5.7 


4.6 



*Treatment group mean is significantly higher (a = ,05) . 
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TABLE AVERAGE WEEKLY PERCENTAGE OF TOTAL READING EXPOSURE 

SPENT IN INDIVIDUAL READING ACTIVITIES IN BASIC 
ELEMENTARY SAMPLE SCHOOLS 



Activity 


Treatment 
(TotaL N = 33) 


Control 
(Total N = 33) 


Total Weekly Reading 
Exposure (in hours) 


X 

10.67 


S.D.' 
2.53 


X 

9.84 


S.D, 

2.25 


Component Activities 


■% 


S.D. 


% 




Group Instruction 


22.5 


7.4 


26.1 


6,0 


Tutpring by Older Person 


6.0 


4.1 


5.0 


2.8 


Peer Tutoring * 


2.7 


1.7 


3.1 


2.2 


Machins-Mediated Instruction 


6.3 / 


3.7 


5.0 


2.3 


Games/Contests 


• 5.-9 




5.3 


2.2 


Diagnostic (Test Taking) 


6.0 


3.0 


5.8 • 


?•! 


Independent Seat Exercises 


19.5 


•4.4 


21.3* 


5.1 


Report/Story Writing 


5.8 


2.4 


6.1 


2.5 


-Individualized Reading \ 


15.4 


5.1 


13.8 


4.8 


Student Presentation \ 

\ 

"Pullout" Compensatory Reading 


3.2 


2.0 


- 3.7 


• 1.9 


6.9 


6.7 


5.8 


5.0 



*Control group mean is significantly higher (a = .05) 
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TABLE A-12. AVERAGE WEEKLY PERCENTAGE OF TOTAL READING EXPOSURE 

SPENT IN INDIVIDUAL READING ACTIVITIES IN BASIC ^ ' Q 
SECONDARY SAMPLE SCHOOLS . \ 



Activity 


Treatment 
(Total N = 11) 


»• Control , 
(Total N = 11 ) 


Total Weekly Reading 
ExDOSUire fin hniir"Q^ 


X 

8,24 


S.D. 
1.12 


X 

7.81^ 


S.D. 
7.81 


Component Activities 


\ 


S.D". 


"% 


S.D. 


Group Instruction 


17.7 


3.4 


16.4 


4.4 


Tutoring by Older Person 


4,9 


2.3 


5.0 


2.2 


Peer Tutoring 


2,2 


1.2 


2.4 


0.9 


Machine->Iediated Instruction 


3.1 


1.4. 


3.2 


1.3 


Games/Co nte stSj. 


5,3 


2.4 


5.7 


2.3 


Diagnostic (Test Taking) 


10.4 


2.5 


9.8 


1.7 


Independent Seat Exercises 


14.2 


1.7 


13.6 


2.3 


Report/Story Writing 


12.2 


3.5 


13.3 


2.3 


Individualized Reading 


19.3 


4.1 


12.3 


6.5 


Student Presentation 


5.9 


2.2 


6.¥ 


1.5^^ 


"Pullout" Compensatory Reading 


5.0 


6.1 


3.2 


0.8 
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TABLE A-13- AVERAGE WEEKLY PERCENT OF TOTAL MATH EXPOSURE SPENT 
IN INDIVIDUAL MATH ACTIVITIES IN PILOT SAMPLE SCHOOLS 





Treatment 


Control 


Unpaired 


Treatment 


Activity- 


^Totai 


M 

N 




^ iocai 


M 
IX 




(Total 


14) 


Total Weekly Math 


V 
A 




n M 

U. li 


Y 
A 


q 


n N 


X 


S.D. N 


Exposure (in hours) 


7,;z.6 


3 


.75 14 


8.30 


4, 


21 14 


8.14 


2.96 14 


Component Activities 


% 




S.D. 


% 




S.D. 


% S.D. 


Group Instruction \ 


29.9 




1.11 


28.0 




1.11 


25.4 


10.8 


Tutoring by Older Person 


' 8.8* 




5.9 


4.8 




3.7 


9.1 


4.6 


Peer Tutoring 


4.8 




3.5 


3.6 




3-2 


3.7 


2,3 


Machine-Mediated 


















4 


3.9\ 




3.8 


4.7 




4.2 


5.3 

• 


3. 3 




8.7 




3.2 


8.3 




3.9 


' 9.3 


4,4 


L/xagnoscxc viesu laNxng; 


8.0 




^2.8 


7-6 




4 • 3 


8.0 


4,1 


Independent Seat Ex^cises 


26.7 




l\ 


22.1 




9.4 , 


21.2 


7.8 


Report Writing 


2.6 




2.5\ 


3.0 




- 3.7 


1.0 


1.0 


Student Presentation 


^.4 




. 3.3 ' 


\5.9** 


4.5 


4.0 


3.1 


"Pullout" Compensatory 








\ 










Math 


2.7 




3.7 


5.7'' 




7.0 


6.0 


6,2 . 



" *Treatment group mean is significantly higher (a = .05). 
**Control group mean is significantly higher (a = .05). 
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TABLE A-14 . AVERAGE WEEKLY PERCENT OF TOTAL MATH EXPOSURE SPENT 
IN INDIVIDUAL iWATH ACTIVITIES IN BASIC ELEMENTARY SAMPLE 

SCHOOLS / 





Treatment 


! Control 




(Ttotal N 


= 33 ) 


(Total N 


= 33 ) 


xOCcll WeeKXy Macn 




X 


S.D. 


X 




Exposure (in hours) 


8 


. 27 


2.79 


'7.77 


2.75 


• 

Component Activities 




% 


S D 


— - 




Group Instruction 


28 


. 3 


i ' 

8.7 / 


30.0 


7.9 


Tutoring by Older Person 


8 


.0 


4.6 1 


6.1 


4,0 


Peer Tutoring 


4 


3 
• -J 


2.8 ; 


4.6 


3.1 


Machine-Mediated* Instruction 


5 


.1 


3.9 ; 


4.5 


3.1 


Games/Con tests 


9 


.0 


3.3 / 


8.5 


2.6» , 


Diagnostic (Test Taking) 


8 


.3 




R R 
o . o 


3 O 


Independent Seat Exercises 


27 


.4 


7.2 


30.4** 


7.7 


Report Writing 


1 


.4 




1.2 


1.1 


Student Presentation 


3 


.8 




3.3 


2.4 


"Pullout" Compensatory Math 


4 


.4* 


i 

/ 


1.9 


2.8 



♦Treatment group mean is significantly higher (a ^ .05) 
**Cont'!rol groUp mean is significantly higher (a =i.05). 
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TABLE A-15. AVERAGE WEEKLY PERCENT OF TOTAL MATH EXPOSURE SPENT IN 
INDIVIDUAL MATH ACTIVITIES IN BASIC SECONDARY SAMPLE SCHOOLS 



Activity 


Treatment 
(Total N = 11 ) 


Control 
.V ' (Total N = 11 ) 


Total Weekly Math 
Exposure (in hours) 


X 
5.84 


S.D. 
1.16 


X 

5.51 


S.D. 
9.7 


Component Activities 


%* 


S.D. 


T 


S.D. 


Group Instruction 








/ . O 


Tutoring by Older Person 


8.4 


3.3 


9.0 


2.2 


Peer Tutoring 


3.5 


1.4 


3-9 


1.0 


Machine-Mediated Instruction 


3.7 


1.6 


4.4 


2.1 


Game s/Con tests 


6.3 


2.9 


6.4 


2.2 


Diagnostic (Test Taking) 


14.5 


4.2 


14.6 


3.4 


Independent Seat Exercises 


21.1 


4.3 


21.5 


4.2 


Report Writing 


4.7 


2.4 


5.8 


3.5 
X 

2.4 


Student Presentation 


6.2 


2,6 


5.4 


"Pullout" Compensatory Math 


7.0 


1,2 


3.4 


1.1 
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TABLE A-16. AVERAGE WEEKLY PERCENTAGE OF TOTAL INTERCULTURAL AND 
COUl^SELING EXPOSURE SPENT Iljl INDIVIDUAL ACTIVITIES 
IN PILOT SAMPLE SCHOOLS 



Activi 



Treatment 
(Total g 14) 



Control 
(Total N = 14) 



Unpaired Treatment 
(Total N =..14) 



Average Total Weekly Inter- 
group Exposure (in hours) 



^ S.D, 
2-48 2.35 



2.32 1,35 



Component Activities 



% 



S,D, 



% 



S-D. 



X 
1-78 



S,D. 
1-21 



Field Trips 

Interracial Programs or 
Projects with Other Schoo 



28-8 2.61 



4,3 5.2 



Cultural Enrichment 
Progrcuns 

ESL or Bilingual 
Activities 

Group Counseling/ 
Guidance 

Individual Counseling/ 
Guidance 



35.8 20,2 



8.2 15,2 



13.5 13.8 



9,4 11.5 



19.7 2.10 



2.3 2,5 



28,5 16,0 



9.2 

20.^ 



11,3 



18,5 



13.4* 10,4 



17.4 16,1 



3.8 



5,2 



34.6 20,9 



8.7 17,2 



19,4 18,5 



8.8 10,6 



r 



*Control group mean is significantly higher (« = .05) 
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TABLE A- 17* AVERAGE WEEKLY PERCENTAGE OP TOTAL INTERCULTURAL AND 
C0U14SELING EXPOSURE SPENT IN INDIVIDUAL ACTIVITIES 
IN BASIC ELEMENTARY SAMPLE SCHOOLS 



1 

Activity 


Treatment 
(Total N = 33) 


Control 
(Tot|,l N = 33) 


Total Weekly Intergroup 
Exposure (in hours) 


X 
2.23 


S.D. ' N 
1.08 33 


X 
1.85 


S.D. N 

1.53 33 


Component Activities 


%" 


S.D. 




S.D. 


« 

Field Trips ^ , 


16.6 


1.69 


16.3 


l.*38 


Int^^cial Programs or 
Projeatts with Other Schpol]S 


4.1 


4.6 


4.9 


5.0 


Cultural Enrichment Programs 


39.0 


' 2.3 * 


41.8 


2..1 


ESL or Bilingual Activities 


5.1 


8.6 


5.1 


1.19 


Group Counseling/Guidance 


20.3 


1.49 


20. 1^-. 


1.59 


Individual Counseling/Guidance 


15.1 


1.41 


11.8 


1.10 



/ 



c 178 

A-18 



TABLE A-18. AVERAGE WEEKLY PERCENTAGE OF TOTAL INTERCULTURAL AMD 
COUNSELING EXPOSURE SPENT IN INDIVIDUAL ACTIVITIES 
IN 'BASIC SECONDARY SAMPLE SCHOOLS 



Activity 


Treatment 
(Total N = 33) 


Control 
(Total N = 33) 


Total Weekly Intergroup 
Exposure (in hours) . 


X 
2.33 


S.D. N 

5.8 10 


X- 
2,45 


S.D. N 

5,7 10 


Component Activities 


% * 


S.D. 


? 


S.D. 


riexd xrips 


23.9 


8.5 


24.9 


6-1 


XII uciTirciu j.cix i jLuy xcuuo or 
Projects with OtR^ Schools 


14.1 


5.6 


r3i.7 


2.6 




15,1 


8.9 


13,3 


5.2 


ESL or Bilingual Activities 


6,7 


2.3 


7.9 


3.2 


Group Counseling/Guidance 


21,3 


8,0 


18.0 


4,3 


Individual Counseling/Guidance 


19,0 


5.6 


22.2* 


4,5 



*Ccntrol group mean is significantly higher (a = ,5) . 
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4J 
rd 
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CD 
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c 
a 
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■H 

c 

rd 
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B 

0 

c 

0 
•H 
4J 
U 
0) 

u 

■H 

13 

(n 
o 

13 

c in 

■H O 



* 
* 



* 



/ 




Program 
& 

Grade 



Pilot; 
3 

' 4 
5 

Basic: 
3 

^ 4 
5 
10 
11 
12 



TABLE A-20. FUNDING-TREATMENT INTERACTION: 
LEVEL OF EACH FUNDING IN THE TREATMENT SCHOOL 



Raw Gam 



M M M 
Voc Comp Cone Comp ^rob 



0.27 0.07 1.04 1.48 
0.13 0.54 0-42 0.18 
1.26 0-14 tl'.34 0.22 



0.46 0-89 0.82 1-15 

0.57 6-58 0.74 0.51 

1.10 0.51 0.94 0:63 

0.16J0.35 0.18 1-74 4.07 

0.21 0.15 0.58^0.15 1.96 

0.09 3.22 0.01 2.48 2.04 



Adjusted Score 



M M M 

Voc Comp Cone Comp*^ Prob 



0.90 0.82 0.29 0.53 
0.39 0.07 0.10 0.62 
0.91 ,0.27 0.12 0.32 



0.35 0.65 0.26 0.65 

1.11 0.45 0,61 0.67 

0.79 0.44 1.69 0. 

1.41 0.12 0.19 1.10 




0.03 0.51 0.35 0.03 ^1.87 
0.03 2.05 3.55 1.17 4.78* 



.05 




) 
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r 



, TABLE FUNDING-TREATMENT INTERACTION: 

TaTAL SUPPLEMENTAL FUNDING LEVEL IN TREATMENT SCHCX)L 



Program 


Raw Gain 


Adiusted ^cbre 








M 


M 


M 






M 


M 


M 


Grade 


Voc 


Comp- 


Cone 


-Comp 


Prob 


Voc 


Comp 


Cone 


Comp 


Prob 


Pilot: 






















3 


0.78 


1.91 


0.57 


0.55 


— - 


0.13 


1.57 


0.16 


0.22 


— 


4 


0.91 


0.70 


0.07 


0.19 




0.81 


1.42 


0.09 


0.05 


— 


5 


0.28 


0.04 


4.56* 


0.95 




0.11 


0.16 


0.69 


0.79 




Basic: 






















3 


0.54 


0.05 


4.68* 


1.54 




1.24 


0.51 


5.33* 


1.81 




4 


0.81 


0.57 


0.59 


0.17 




0.34 


0.80 


0;48 


0.25 




5 


2.46 


0.77 


1.44 


1.30 




0.97 


0.97 


1.99 


1.12 




io 


2.92 


0.33 


0.14 


0.54 


1.18 


2.93 


0.05 


0.26 


0.62 


2.75 


11 


0.01 


0.51 


1.34 


5.05* 


1.96 


0.04 


0.08 


1.02 


10.23* 


2.42 


12 


•0.47 


1.98 


0.55 


0.14 


0.37 


0.42 


1.53 


0.34 


5.96* 


0.41 
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TABLE A-22. FUNDING-TKEATMENT INTERACTION: 
DIFFERENCE BETWEEN TREATMENT-CONTROL TOTAL SUPPLEMENTAL FUNDING 



Program 
Grade 



Rav/ Gain 







M 


^M 


Voc 


Comp 


Cone 


Comp 


2.08 


2.86 


0,76 


• 


2.87 


1.58 


0,90 


0.12 


1.77 




■^^52 


1.64 



Adjusted Score 



M M M' 

Voc Comp Cone Comp ' Prob 



Pilot: 




^^asic ; 



3 


0.67 


1-41 


4 


0.52 


0.47 


5 


1.09 


0.15 


10 


0.05 


0.33 


11 


0.44 


0.00 


12 


0.02 


1.74 



0,57 5^.35? ~ 
j^,04 2.41 ' ~ ! 
0.??^ 1.35 " - 

0,02 1.60 2,75 
i 0,j7|7 0j*47 1.45 

0137 ^.37 2.03 



0,77 2.13 0,44 0.28 



•3.75* 2.48 0,92 0.11 



1,68 2,78 0,69 1.93 



0,67 1,46 0,24 4.04* 

1.01 1,08 1,90 3,29* 

1*98 0.3^ 0,84 2,12 

0,73 0,10*^ 0,05 0,87. 

0.22\ 0,62 0,18 1,10 

0,17 ^2,70 0,20 9,68 



1,58 
0,72 
.0,19 



*p < , 05 
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TABLE A-23. FUNDING TREATMENT INTERACTION: DISTRICT REGULAR FUNDING 



^ 



Program 
Si 

Grade 



Raw Gain 



M M, 
Voc Comp Cone Comp Prob 



Adjusted ^core 



Voc Comp 



M 

Cone 



M 

Comp 



M 

Prob 



Pilot: 
3 
4 
5 

Basic: 
3 
4 
5 



10 
11 

* P < .05 



0.33 0.67 

0.71 1.19 

1.18 1.20 

0.49 0.16 

0.56 0.08 

1.23 3.09 



0..26 0.14. 0,14 7.39* — 
0.74 1.96 ^0.85 1.28 — 
0.42 1.63 0.30 0.43 — 



0.'59 7.06* — - 



0.O7 1.26 — 

2.81* 0.54 ~ 

I 

0.64 . 0.70 00 \ 

1.84 0.68 0.01 
0.19^ 0.24 0.14 . 

1.85 5.03* 1.33 



(X^l^. 
0^.22 0.21 0.50 1.38 
0.47 1.51 1.22 0."38 



0.61 1.53 0.27 2.12 
0;95 1.33 2.30 0.-34 
1.44 0.44 0.73 .0.47 



3.46 ^0.16 Trn — ""0773 0704 
0.36 0.15 O.dl 0.29 ^ 0.23- 
2.03 0.25 0.20 3.58 0.53 
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TABLE A-24. EXPOSUte-TREATMENT INTERACTION: T-C READING EXPOSURE DIFFERENCE 



Program 
& 

Grade 



Raw Gain 



Voc Comp Cone 



M I M 
one 



M 



Pilot: 














3 


0, 13 


3, 66^ 


1-19 


0.04 






4 


0.99 


3-18 


0,23 


0.50 






5 


0.13 


0,82 


1,82 


1.^69 






Basic: 














<[> 


1-39 


1-40 


0,82 


0.75 






4 


0.11 


0-67 


1,37 


0,41 






5 


1-58 


2,02 


0,66 


0,79 






10 


0.40 


0-04 


12,95* 


0.73 


0, 


26 


11 


8.11* 


2.31 


0,72 


1.02 


0, 


37 


12 


0-89 


0-17 


1-22 


0.49 


0, 


62 



Adjusted Score 



M M M 

Voc Comp .Cone Comp Prob 



0,06 * 1.79 1,33 ^ 0,23 
0,27 2.35 . 0,09 , 0,95 



0.32 0.44 1.43 AL,38 



0,39 3.51* 0,84 1,53- 
0,46 0.93 /,64 0,62- 
2,40 2.51 'l,31 1,45 



0,04 0.33 5,86* 0,71 
L0,18* 7.34* 0.83 0.76 
0,B9 0.51 0.0<S^ 0,50 



\ 



O.09 
0.34 
0.45 



* P .05 
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Program 


Raw Gain 


Adiusted Score 


Si 






M 


M 


\ M 






M 


M 


M 


Grade 


Voc 


Comp 


Cone 


Comp 


Prob 


Voc 


Comp 


Cone 


Comp 


Prob 


Pilot: 
























0.29 


1.67 


0^76 


0.12 


— 


0..19 


0.23 


0.28 


0.24 


— 


^ 4 


0.71 


1.03 


0.13 


0.42 


— 


0.29 


1.03 


0.31 


0.56 


— 


5 


0.13 


0.82 


1.82 


1,69 




0.32 


0.44 


1.43 


1. 38 


— 


Basic: 
























0.61 


0.93 


1.15 


0.30 




0.82 


0.41 


0.76 


0.51 




4 


0. 38 


0.17 


0.36 


0791 




, 0.61 


0.51 


0.21 


1.08 




5 


% 

1 . 




U . 17 


U .41 




fD 


1 . DO 


\J . HO 


\J • O f 




10 


2.74 


0.05 


3.59 


0.26 


0.35 


1.23 


0.38 


2.16 


0.37 


0.12 


11 . 


0.56 


0.29 


0.60 


0.08 


2.30 


0.22 


0.24 


1.76 


0.18 


2.10 , 


12 , 


0.05 


0.06 


■0.15 


0.02 " 


0.59 


0.03 


0.22 


0.66 


3.78 


0.10 




TABLg A-26. PROGRAM FOCUS-TREATMENT INTERACTION: WORD TECHNIQUES (VOCABULARY) 



1 



Program 


Pwaw Gain 


Adiusted Score 


& 






M 


M 


M 






M 


M 




Grade 


Voc 


Comp 


Coni 


Comp 


Prob ' 


V \J\^ 




Cone 


Comp 


Prob 


'D i 1 • 






















"3 


!• 30 


2.13 


* 0 19 


n no 


• 


yj * 










4- 


0*27 


0.35 


2.23 


2.41 




, 1.28 


1.34 


..2.68 


4.94* 






0.66 


1.92 


0.57 


0.16 


— 


0.79 


1.98 


0.30 


0.13 




Basic: - 






• 
















•J 


0*22 


1.03 


5.92* 


1.26 




0.31 


0.66 


6.39* 


1 .38 




4 




0.71 


1.10 


i.49 


- — 


b.42 


0.38 


1.15 . 


1.33> 


— 


5 


2.37 


1.27 


2.16 


3.19* 




2.27 


3.04* 


4.10* 


3.60* 




10 


0.04 


0.30 


1.52 


1.16 


0.96 


0.11 


0.62 


1.28 


1.17 


0.30 


11 


0.16 


0'.69 


1.14 


0.24 


1.33 


0.08 


1.28 


0.69 


0.21 


0.88 
















i.oi 






^.02 


n- 1 


0.90 


1.94 


0.02 


0;05 


0.77 


1.65 


0.85 


7.49* 



* P < .05 




TABLE A-27. PROGRAM FOCUS-TREATMENT INTERACTION: READING COMPREHENSION 



Program 


Raw Gain 


Adiusted Score 


Grade 


Voc 


Comp 


M 

Cone 


M 
Comp 


M 

Prob 


Voc 


Comp 


M 
Cone 


M 

Comp 


M 
Prob 


Pilot: 
























3 


0.04 


0.03 


0.03 


0.07 


— 


0.00 


0.11 


0.03 


6.00 


— 


4 


0.65 


1.46 


0.63 


0 


.49 


— 


1.29 


2.70 


0.56 . 


1.06 


— 


5 


1.11 


0.53 


3,46 


0 


.77 


— 


1.56 


0.02 


1.38 


0.73 


— 


BcisiLc * 
























3 


0.60 


0.25 


0.33 


0 


.83 


— 


0.58 


0.25 


0.90 


0.69 


— 


4 


0.12 


0.64 


0.83 


0 


i 

|41 


— . 


0^35 


0.05 


0.43 


0.68 


— 


5' 


1.16 


1.40 


0,.48 


0 


j64 




1.36 


1.17 


0.66 


0*63 




10 


0.76 


O.Ol 


1.69 


0 


.03 


1.37 


0.00 


0.75 


1.76 


0.08 


1.69 


11 


0.29 


0.99 


1.70 


0 


.37 


0.73 


0.08 


0.33 


0.61 


1.00 


1.17 


12 


3-30 


1.15 


0.18 


0 


.01 


1.84 


2.03 


1.15 


1.73 


1.29 


4.*45 
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A-28 



TABLE PROGRAM FOCUS -TREATMENT! INTERACTION: MATH CONCEPTS 



Program 


Raw Gain 


AdTUsted Score / I 


& 

Grade 


Voc * 


Comp 


M 
Cone 


Comp 


M I 
Probl 


Voc 


Comp 


M 
Cone 


M 

Comp 


Prob 


Pilot: 












\ 










3 


0.05 


0.49 


0.01 


0.45 


\ 


0.01 


1.12 


0.02 


1.04 


__ 


4 


0.00 


0.01 


0.05 


0.05 




0.59 


0.47 


0.01 


0.52 


„ 


5 


2.64 


1.14 


0.21 


0.15 




7.40* 


0.03 


0.35 


0.41 


— 


Basic: 










I 












3 


0.17 


2.49 


0.81 


0.18 




0.59 


2.63 


0.68 


1.05 ' 




4 


0.58 


1.02 


0.72 


1.44 


i 


0.35 


iroo 


,1.21 


2.11 


— 


5 


0.68 


0.43 


0.47 


0.86 




0.62 


0.40 


0.11 


0.55 




10 


0.58 


2.86 


1.71 


2.20 


6.73 


1.37 


0.65 


3.59 


4.06 


0.03 




1.23 


3.68 


1.27 


0.01 


b.io 


0.75 


3.47 


4'>J.5 


0.00 


0.08 


12 


0.70 


0.26 


0.81'' 


1.32 


jO.89 


0.79 


0.48 


0.00 


0.02 


0.62 



* P < .05 



1 
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TABLE A-29, PROGRAM FOCUS -TREATMENT INTERACTION: MATH OPERATIONS 



Program 


Raw Gain 


Adi us ted Score 


Grade 


Voc 


Com^ 


M 

Cone 


M 
Comp 


M 

Prob 


Voc 


Comp 


M 

Cone 


M 
Comp 


M 

^Prob 


Pilot: 
























0.43 


2.83 


1,33 


1.05 


— 


0.15 


0.05 


2.77 


2.62 


— 


4 
5 






















Basic: 






















3 


2.32 


1.34 


2.37 


2.01 


— 


1.62 


0.97 


3.86* 


1.20 





4 


2.01 


0.79 


0.06 


2.57 


— 


4.16* 


2.41 


1.56 


3.42* 





5 


0.43 


0.04 


0.54 


0.31 




0.71 


0.31 


0.06 


0.20 




10 


0.02 


0.03 


4.85 


8.28* 


0.38 


0.07 


0.41 


4.10 


10.54* 


0.02 


11 


0.79 


5.67* 


2.09 


0.01 


0..14 


1.26 


5.68* 


^.12 


0.00 


0.12 


12 . 


0.46 


0.37 


0.10 


4.85 


1.20 


1.46 


0.01 


1,53 


0.15 


2.56 



* P < .05 
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A»ENDIX 



ESM NATIONAL EVALUATION ADVISORY PANEL'S 
STATEMENT OF INVOLVEMENT 



(1) Research Design: 



Since its inception in February ^r^^ 197 3 , the ESAA evaluation has had the benefit 
of advice fron%a number of individuals in four fields, namely: 

Dr. William Coffman — University of Iowa 
Dr. Seymour Feshbach — University of California 

at Los Angeles 
Dr. Chester Harris — University of California 

at Santa Barbara 
Dr. Melvin Novick — University of Iowa 
Dr. David Wiley — University of Chicago 



(2) Test Selection: 



Dr. Nancy Cole — University of Pittsburgh 
Dr. Robert Hess — Stanford University 
Dr. Charles Thomas — Indiana University 
Dr. Ralph Tyler — Center for Advanced Study In 
Behavioral Sciences 



(3) Equal Educational 
Opportunity : 



Dr. Thomas Pettigrew— Harvard University 
Professor Meyer Weinberg — City College of 
Chicago 



(4) Meta Evaluation; 



Dr. Gene Glass — University of Colorado \^ 
Dr. Michael Scriven — University of California 

at Berkeley ! 
Dr. Daniel Stufflebeam — Western Michigan 

University 



During the first opex^ating year (1973-74) , criticism by organizations and indi- 
viduals of the "Race Relations" questionnaire then in use resulted in with^irawal 
of the in.strument. A "Blue Ribbon Panel" was then established to advise USOE 
and SDC on this aspect of the evaluation. ^ 
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USOE asked several organizations to name panel representatives. The following 
were chosen by their respective organizations to be members of the panel: 



Mr. Iiuis Alvarez 
ASPIRA of America, Inc. 

Dr. Boyd Bosma 

National Education Association 

Dr. Buell Gallagher 
National Association for the 
Advancement of Colored People 

Dr. Joseph Garcia and 
Mr. Tony Vasquez 
National Education Task Force 
de la Raza 

Dr. Robert Hill 
National Urban League 

Dr. Thomas Hi-lliard 
(replaced by Dr. George Jadkson ) 
and Dr. Ruth King 
National. Association of Black 
Psychologists 



Mr. Sam Husk 

Council of 'Great City Schools 

Mr. Lemuel Ignacio 
Pacific Asian Coalition 

Dr. Jofths^tiglmeier 

(replaced by Dr. Jess Elliott) 

Chief School Officers 

Committee on Evaluation and Information 
Systems 

Dr. Charles Townsel 
National Alliance of Black School 
Educators 



In addition, the following individuals were chosen by USOE: 



Dr. Edgar Epps 
University of Chicago 

Dr. Charles Clock 
University of California 
, at Berkeley 

Dr. Patricia X3urin 
University of Michigan 



Dr. Ronald Milivasky 

National Broadcasting Corporation 

Mrs. T^kako Okubo 
Elementary Teacher 

Mr. Paul Sheatsley 

National Opinion Research Center 



This second advisory group had four three-day meetings between March and Atigust, 
1974, and submitted critical comments and ^suggestions regarding various measures 
of intergroup relations. A new instrument on "School Climate" was constructed^ 
to replace the "Race Relations" questionnaire. The second year of the evaluation 
(1974-75) also reflects some adjustments in procedure, method, and. purpose as 
a result of "Blue Ribbon Panel" participation. 
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AtJthe November 1974 meejting of the evaluation advisors, the original panel 
mdmbers and the "Blue Ribbon Panel" (with a few changes in personnel) were 
btought together to form a single pSAA National Evaluation Advisory Panel, 
>whose members continue to offer critical advice as the evaluation goes fo;cward^ 
y This newlypcohstituted combined panel advises on all aspects of the evaluation 
project, including this and otiier reports. 

The responsibility for this present report on the year 1973-74, as well as for 
the continuing evaluation process and its results, rests with the Office of 
Education and \ts contractor. System Development Corporation, not with the 
advisors. Nevertheless, after reviewing an early draft of this report, the 
panel as a whole has developed a statement which appears in Appendix C. 

One advisor holds views about the evaluation project and this report Jihich he 
believes to be of sufficient importance to warrant publication at this 5^tage 
of the continuing evaluation. The views of Dr- George Jagkson appear irk;- 
Appendix D of this report. 
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APPENDIX C 



STATEMENT OF ESAA NATIONAL EVALUATION ADVISORY PANEL* 



In November 1974 ^ the ESAA National Advisory Panel adopted the following 
statement: 

It: is the unanimous recommendation of the National Evaluation Advisory Panel 
that the first-year report by System Development Corporation of the ESAA Basic 
and ESAA Pilot evaluation focus on the descriptive aspects of the findings. 
Specifically/ the report should describe , characteristics of the programs that 
have been funded and implemented, characteristics of - the student ^population 
that has participated in ESAA funded and matched control schools, and charac- 
teristics of the respective school' staffs and administrators. 

It: is also the unanimous view of the National Evaluation Advisory Panel that 
an "imn;^f:t" and "relational" analysis of the effects of the ESAA program on 
changes in reading achievement and mathematics achievement after less than six 
months of ESAA program implementation is premature. It is Ijighly improbable 
that educational programs that havf been in operation for so "short a period of 
time could produce significant differential increments in academic skills. 

TJie National Evaluation Advisory Panel is deeply concerned that the highly- 
probable outcome of a first-year evaluation finding of "no significant 
differences" as a function of ESAA funding will be misinterpreted and misread 
as a commentary on the eff --ctiveness of the ESAA program. The analyses of the ^ 
first-year pretest and posttest achievement scores should be primarily address- 
ed to the establishment of reliable base rates of performance. - 

Changes in academic achievement, in desegregation practices, in student and 
teacher attitudes, and in related outcome areas can be meaningfully evaluated 
only after a sufficient time has elapsed for ESAA funding to exert an impact. 
In this regard the analysis of educational changes after a two-year period of 
ESAA funding has scientific meaning and relevance. However, even more critical 
is the developmental analysis of educational changes that is planned for the 
third year of program evaluation. 

The National Evaluation Advisory Panel wishes to underline the importance of a 
third year of funding in order to accomplish a meaningful evaluation of program 
impact and educational changes. The history of evali^^ation of federally- funded 



It should be noted that this statement, was drafted aftqr review , of an early 
draft of this report and has not been revised to reflect changes made since then. 
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'programs has too often been characterized by premature conclusions based upon 
the limited' educa.tional effects of programs that have been in operation for 
less than two years, or even as little as one year* ^ ' 

ESAA programs* were never' viewed as magic elixirs that would produce dramatic 
transformations in achievement levels. To be fairly evaluated, the programs 
require a fair opportunity for implementation and for producing measurable 
e^ucationfal consequences. 

After reviewing a draft of the first-year report in June, 1975, the Advisory 
Panel expresses the following concerns: 

1, Because of the relatively short perioc? of time for ESAA program implementa- 
tion within the first year, we reaffirm our earlier position that it is premature 
to expept significant gains in standardized test scores among the study sample 
due to '^ISAA funding* 

2. Because of the relatively small differences in supplemental funds to ESAA- 
funded ahd comparison schools, it is also not reasonable to expect significant 
gains in standardized test scores as a result of ESAA funds^ 

3* Because of the eiupuasls in the tirst-year report^ on. test scores as the 
primary criteria for program success and impact, we are disturbed that in- 
adequate attention has been given to the importance of intervening or process 
variables, such as the attitudes and other characteristics of students and 
teachers, desegregation-related measures, other school climate measures, and 
length of exposure to different ESAA programs, as' significant program correlates 
or measures of program success and impact. ^ * 

Although we are aware that the lack of sufficient pre- and post-measures of 
program outcomes other than test scores is a serious constraint on satisfying 
these concerns, we still strongly urge that the highest priority be given to 
the incorporating of intervening process variables as significant dependent 
variables in this as well as in subsequent reports. 

4. We are concerned that there has been insufficient in-depth analysis of 
program impact and correlates separately for students from different ethnic 
groups, specifically. Blacks, Spanish- speeiking and Whites relating to measures 
other than test scores. 

5. _ While we are, aware that descriptive program information is being gathered 
at selected local sites, we feel that there is an urgent need to incorporate 
more of the descriptive local program data into the first-year report. 

6. We think that special caution must be exercised with regard to the 
generalizability of particular findings in this preliminary first-year report 
because of severe, constraints in the study design and methodology, specifically 
the reliance on standardized test Scores and the use of national norms* even 
as baseline data, for students in the ESAA study sample. 
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CONCLUSIONS 

The Panel wishes to acknowledge the assistance and cooperation 'of .SDC in 
assisting, the panel's efforts and in providing an opportunity for input 
from the panel. The recommendations and concerns expressed by that panel 
are primarily addressed to problems inherent in evaluations 6f this , scope 
and magnitude rather than to the efforts of the research contractor. 
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APPENDIX D 

POSITION STATEMENT ON YEAR ONE REPORT—ESAA EVALUATION PROJECT, 
PREPARED BY PR, GEORGE D, JACKSON, PH,D, , CHAIRMAN, 
NATIONAL ASSOCIATION OF BLACK PSYCHOLOGISTS > 

June 19, 1975 

While the Advisory Panel continues to ba critical of the ESAA 
Evaluation Project, the National Association of Black Psychologists 
takes a stronger postiire. The Association deems it essential to 
state for the record its unequivocal opposition to this project. It 
does not endorse the Year One Report, 

Among the reasons for non-sndofsement are the following: 

1) The use of an experimental design dravm from the animal 
paradigm is immoral and inapplicable to the study of human achievement 
in the educational enterprise. Strong exception is taken to the use 
of money as an independent variable. The deliberate withholding of 
those which allegedly money could buy, apparently without consent of 
the affected persons, is inhumane and unconscionable^. No justifica- 
tion, not even that of "limited funding" can alter the moral issue. 

\^ Recognition of the inapplicability, minimally at least 

at^^the pragmatic level, of this design is evidenced at the district 
level by the action of some superintendents who distributed money from 
other\sources to the "control" schools. In this way they sought to 
provide' equity for their students, but, in effect, "washed out" the 
design. ^ The project reaction to this was to perform statistical mani- 
pulations aimed at determining the relationship between achievement 
and level of funding (if any). These manipulations, we contend, gave 
rise to spurious data. 

'2) Wo cannot endorse a study which employs as its dependent 
measure a standardized instrument which has been shown to be invalid * 
for Blacks and other minpriiies. We' also contend that restandardiza- 
tion assumes that the learning of pluralistic groups, is referable to 
the normal curve. This assumption is of doubtful validity even when ■ 
dealing with a homogeneous population. With a pluralistic society 
this assumption is untenable. 

3) The lack of Latino and other non-Black minorities in this 
study makes it impossible to discuss minority isolation and, other 
related issues in a social-scientif ically valid manner* It suggests 
that an invidious form of racism pervades the ESAA Evaluation Project. 
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4) Data regarding activity of large cities is unavailable and, 
thus, this study has di\bious merit inflight of the weighty issues in 
contemporary Urban education. 

'5) The assumptions revealed in the discussions on program 
suggest that the evaluators are not conversant with contemporary 
pedagogy. For example, the laboratory approach to mathematics is 
described as new; small groups are seemingly equated with individ- 
ualization; categorization of remedial reading techniques contains 
inadequate di-fferentiation. 

6) The report contains a nuinber of procedural statistical 
errors and questioneible assumptions. In general it lacks the scholar- 
ship which we would expect to be present in a document which potends 
to have an effect on thousands of our young people. 

For example, expected achievement for minorities is based on 
an anonymous memorandum and/or arbitrary readjustment of national 
norms. A more accurate picture could have been obtained by measur- 
ing students against their own baselines and along more than the 
two variable, reading and mathematics.' 

The ubiquitous use of percentiles and percefnts tends to obscpre 
numbers and individuals.^ For example, if in a group of lOO people, 
99 get a score of 90 or better anJ only one scores 89, the latter is 

'classed in the 1st percentile. Also, contemporary remediation, which 
is a purported purpose of ESAA funding, tends to stress individualiza- 
tion and success of individuals. The prevalent use of percentages in 

-this report tends to obscure this dimension. 

Among other things, the small amount of funding received by some 
Basic/Pilot districts relative to the amount actually allocated to the 
area (e.g. New York) calls into question the generalizability of the 
results, even if they had been validly attained. 

The Association of Black Psychologists recognizes that the above 
judgments were based on. the first draft of the Year One Report. How- 
ever, while changes may be made ifi the final draft, they will not alter 
the substantive objections which we have made above. 

It i4 not our intention to impugn educational projects which may 
have been initiated with ESAA money. Moreover, it is recognized that 
some alterations in evaluative procedures may need to occyr during the 
life of a project and should be made with scientific care. In this 
instance, however, the project design itself vitiates its stated purpose 
and, we contend, requires massive alteration. 

t 
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At this point, oiir ♦position must be made clear in the hope that 
this project and/or other attempts to evaluate educational activity 
with respect -to minorities will avoid the many errors so vividly 
present here* 
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